[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:33:01 dist-test-slave-vg89 krb5kdc[15696](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:33:01 dist-test-slave-vg89 krb5kdc[15696](info): set up 2 sockets
Sep 29 10:33:01 dist-test-slave-vg89 krb5kdc[15696](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260929 10:33:02.683103 15689 mini_kdc.cc:121] Time spent starting KDC: real 1.046s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Sep 29 10:33:02 dist-test-slave-vg89 krb5kdc[15696](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790677982, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:33:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:33:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.82.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab.
Entry for principal kudu/127.15.82.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab.
I20260929 10:33:02.816804 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:39153
--webserver_interface=127.15.82.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab
--principal=kudu/127.15.82.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:46749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.82.126:39153
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:33:02.929031 15709 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:33:02.929349 15709 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:33:02.929432 15709 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:33:02.933531 15709 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:33:02.933629 15709 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:33:02.933669 15709 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:33:02.933732 15709 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:33:02.933771 15709 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:33:02.939216 15709 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:46749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.82.126:39153
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab
--principal=kudu/127.15.82.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:39153
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.82.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15709
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:33:02.940317 15709 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:33:02.941375 15709 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:33:02.950609 15714 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:02.950625 15717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:02.950695 15715 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:02.951089 15709 server_base.cc:1034] running on GCE node
I20260929 10:33:02.951628 15709 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:33:02.952713 15709 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:33:02.953909 15709 hybrid_clock.cc:648] HybridClock initialized: now 1790677982953898 us; error 32 us; skew 500 ppm
Sep 29 10:33:02 dist-test-slave-vg89 krb5kdc[15696](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790677982, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:33:02.956638 15709 init.cc:377] Logged in from keytab as kudu/127.15.82.126@KRBTEST.COM (short username kudu)
I20260929 10:33:02.957603 15709 webserver.cc:460] Webserver started at http://127.15.82.126:44967/ using document root <none> and password file <none>
I20260929 10:33:02.958161 15709 fs_manager.cc:359] Metadata directory not provided
I20260929 10:33:02.958250 15709 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:33:02.958540 15709 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:33:02.960608 15709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/minicluster-data/master-0/data/instance:
uuid: "1ff07bbce69a4752a05371b068a57104"
format_stamp: "Formatted at 2026-09-29 10:33:02 on dist-test-slave-vg89"
server_key: "f0e3577032fb8b64773db3662e67764a"
server_key_iv: "5aaa8f6fcb07c0eb4cd2f3ebdc3af123"
server_key_version: "encryptionkey@0"
I20260929 10:33:02.961184 15709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/minicluster-data/master-0/wal/instance:
uuid: "1ff07bbce69a4752a05371b068a57104"
format_stamp: "Formatted at 2026-09-29 10:33:02 on dist-test-slave-vg89"
server_key: "f0e3577032fb8b64773db3662e67764a"
server_key_iv: "5aaa8f6fcb07c0eb4cd2f3ebdc3af123"
server_key_version: "encryptionkey@0"
I20260929 10:33:02.965046 15709 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260929 10:33:02.967463 15724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:02.968675 15709 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 10:33:02.968811 15709 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/minicluster-data/master-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/minicluster-data/master-0/wal
uuid: "1ff07bbce69a4752a05371b068a57104"
format_stamp: "Formatted at 2026-09-29 10:33:02 on dist-test-slave-vg89"
server_key: "f0e3577032fb8b64773db3662e67764a"
server_key_iv: "5aaa8f6fcb07c0eb4cd2f3ebdc3af123"
server_key_version: "encryptionkey@0"
I20260929 10:33:02.968919 15709 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:33:03.002461 15709 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:33:03.005617 15709 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:33:03.005828 15709 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:33:03.013926 15709 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.126:39153
I20260929 10:33:03.013952 15776 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.126:39153 every 8 connection(s)
I20260929 10:33:03.014918 15709 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
I20260929 10:33:03.017776 15777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:33:03.023432 15777 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ff07bbce69a4752a05371b068a57104: Bootstrap starting.
I20260929 10:33:03.024242 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 15709
I20260929 10:33:03.024407 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/minicluster-data/master-0/wal/instance
I20260929 10:33:03.026101 15689 external_mini_cluster.cc:1428] Setting key dac97d5a18d1a14e5d17994c044d5c60
I20260929 10:33:03.026724 15777 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ff07bbce69a4752a05371b068a57104: Neither blocks nor log segments found. Creating new log.
I20260929 10:33:03.027745 15777 log.cc:824] T 00000000000000000000000000000000 P 1ff07bbce69a4752a05371b068a57104: Log is configured to *not* fsync() on all Append() calls
I20260929 10:33:03.029989 15777 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ff07bbce69a4752a05371b068a57104: No bootstrap required, opened a new log
I20260929 10:33:03.034262 15777 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ff07bbce69a4752a05371b068a57104 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ff07bbce69a4752a05371b068a57104" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 39153 } }
I20260929 10:33:03.034559 15777 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1ff07bbce69a4752a05371b068a57104 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:33:03.034631 15777 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ff07bbce69a4752a05371b068a57104 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ff07bbce69a4752a05371b068a57104, State: Initialized, Role: FOLLOWER
I20260929 10:33:03.035166 15777 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ff07bbce69a4752a05371b068a57104 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ff07bbce69a4752a05371b068a57104" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 39153 } }
I20260929 10:33:03.035333 15777 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1ff07bbce69a4752a05371b068a57104 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:33:03.035439 15777 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ff07bbce69a4752a05371b068a57104 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:33:03.035571 15777 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ff07bbce69a4752a05371b068a57104 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:33:03.036512 15777 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ff07bbce69a4752a05371b068a57104 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ff07bbce69a4752a05371b068a57104" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 39153 } }
I20260929 10:33:03.036878 15777 leader_election.cc:302] T 00000000000000000000000000000000 P 1ff07bbce69a4752a05371b068a57104 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ff07bbce69a4752a05371b068a57104; no voters:
I20260929 10:33:03.037197 15777 leader_election.cc:288] T 00000000000000000000000000000000 P 1ff07bbce69a4752a05371b068a57104 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:33:03.037312 15782 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ff07bbce69a4752a05371b068a57104 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:33:03.037571 15782 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ff07bbce69a4752a05371b068a57104 [term 1 LEADER]: Becoming Leader. State: Replica: 1ff07bbce69a4752a05371b068a57104, State: Running, Role: LEADER
I20260929 10:33:03.037945 15782 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ff07bbce69a4752a05371b068a57104 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ff07bbce69a4752a05371b068a57104" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 39153 } }
I20260929 10:33:03.038535 15777 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ff07bbce69a4752a05371b068a57104 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:33:03.039789 15783 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ff07bbce69a4752a05371b068a57104 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1ff07bbce69a4752a05371b068a57104" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ff07bbce69a4752a05371b068a57104" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 39153 } } }
I20260929 10:33:03.039925 15783 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ff07bbce69a4752a05371b068a57104 [sys.catalog]: This master's current role is: LEADER
I20260929 10:33:03.040339 15784 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ff07bbce69a4752a05371b068a57104 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ff07bbce69a4752a05371b068a57104. Latest consensus state: current_term: 1 leader_uuid: "1ff07bbce69a4752a05371b068a57104" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ff07bbce69a4752a05371b068a57104" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 39153 } } }
I20260929 10:33:03.040508 15784 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ff07bbce69a4752a05371b068a57104 [sys.catalog]: This master's current role is: LEADER
I20260929 10:33:03.040874 15791 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:33:03.044687 15791 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 29 10:33:03 dist-test-slave-vg89 krb5kdc[15696](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790677982, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:33:03.051069 15791 catalog_manager.cc:1347] Generated new cluster ID: 82ce48e3e8d94d50b6432ca12f7a9e45
I20260929 10:33:03.051126 15791 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:33:03.052487 15780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:03.029710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54992 (local address 127.15.82.126:39153)
0929 10:33:03.030172 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:03.030191 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:03.031398 (+ 1207us) server_negotiation.cc:407] Connection header received
0929 10:33:03.034080 (+ 2682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:03.034109 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:03.034519 (+ 410us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:03.034921 (+ 402us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:03.037614 (+ 2693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:03.039264 (+ 1650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:03.041295 (+ 2031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:03.042000 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:03.042158 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:03.046037 (+ 3879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:03.046042 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:03.046054 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:03.046081 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:03.048269 (+ 2188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:03.048937 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:03.048945 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:03.048956 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:03.049067 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:03.049517 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:03.049521 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:03.049523 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:03.050061 (+ 538us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:03.050322 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:03.051502 (+ 1180us) server_negotiation.cc:299] Negotiation successful
0929 10:33:03.051810 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":116,"threads_started":1}
I20260929 10:33:03.075500 15791 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:33:03.076439 15791 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:33:03.082641 15791 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1ff07bbce69a4752a05371b068a57104: Generated new TSK 0
I20260929 10:33:03.083413 15791 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.82.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab.
Entry for principal kudu/127.15.82.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab.
I20260929 10:33:03.121846 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.65:0
--local_ip_for_outbound_sockets=127.15.82.65
--webserver_interface=127.15.82.65
--webserver_port=0
--tserver_master_addrs=127.15.82.126:39153
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab
--principal=kudu/127.15.82.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:46749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:33:03.257422 15803 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:33:03.257709 15803 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:33:03.257787 15803 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:33:03.261888 15803 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:33:03.261971 15803 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:33:03.262063 15803 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.65
I20260929 10:33:03.267521 15803 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:46749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab
--principal=kudu/127.15.82.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.82.65
--webserver_port=0
--tserver_master_addrs=127.15.82.126:39153
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15803
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.65
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:33:03.268676 15803 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:33:03.269685 15803 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:33:03.279139 15811 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:03.279172 15809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:03.279168 15808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:03.279496 15803 server_base.cc:1034] running on GCE node
I20260929 10:33:03.279894 15803 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:33:03.280580 15803 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:33:03.281764 15803 hybrid_clock.cc:648] HybridClock initialized: now 1790677983281741 us; error 39 us; skew 500 ppm
Sep 29 10:33:03 dist-test-slave-vg89 krb5kdc[15696](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790677983, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:33:03.284443 15803 init.cc:377] Logged in from keytab as kudu/127.15.82.65@KRBTEST.COM (short username kudu)
I20260929 10:33:03.285316 15803 webserver.cc:460] Webserver started at http://127.15.82.65:43463/ using document root <none> and password file <none>
I20260929 10:33:03.285866 15803 fs_manager.cc:359] Metadata directory not provided
I20260929 10:33:03.285952 15803 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:33:03.286177 15803 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:33:03.288209 15803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/minicluster-data/ts-0/data/instance:
uuid: "32ed1c1730974c46a546d13973f8be84"
format_stamp: "Formatted at 2026-09-29 10:33:03 on dist-test-slave-vg89"
server_key: "4d5ac6e108d4af26768e325534fa3f7d"
server_key_iv: "b2a3df64458650814be96d05421929c1"
server_key_version: "encryptionkey@0"
I20260929 10:33:03.288826 15803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/minicluster-data/ts-0/wal/instance:
uuid: "32ed1c1730974c46a546d13973f8be84"
format_stamp: "Formatted at 2026-09-29 10:33:03 on dist-test-slave-vg89"
server_key: "4d5ac6e108d4af26768e325534fa3f7d"
server_key_iv: "b2a3df64458650814be96d05421929c1"
server_key_version: "encryptionkey@0"
I20260929 10:33:03.292625 15803 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 10:33:03.294977 15818 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:03.296165 15803 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 10:33:03.296289 15803 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/minicluster-data/ts-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/minicluster-data/ts-0/wal
uuid: "32ed1c1730974c46a546d13973f8be84"
format_stamp: "Formatted at 2026-09-29 10:33:03 on dist-test-slave-vg89"
server_key: "4d5ac6e108d4af26768e325534fa3f7d"
server_key_iv: "b2a3df64458650814be96d05421929c1"
server_key_version: "encryptionkey@0"
I20260929 10:33:03.296393 15803 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:33:03.306887 15803 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:33:03.309720 15803 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:33:03.309908 15803 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:33:03.310505 15803 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:33:03.311264 15803 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:33:03.311332 15803 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:03.311401 15803 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:33:03.311443 15803 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:03.320379 15803 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.65:36941
I20260929 10:33:03.320407 15931 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.65:36941 every 8 connection(s)
I20260929 10:33:03.321255 15803 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
I20260929 10:33:03.329147 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 15803
I20260929 10:33:03.329262 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/minicluster-data/ts-0/wal/instance
I20260929 10:33:03.329564 15689 external_mini_cluster.cc:1428] Setting key 6770eccb22fe850c5ca4187f1ed01557
Sep 29 10:33:03 dist-test-slave-vg89 krb5kdc[15696](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790677983, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.65@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:33:03.335162 15780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:03.323065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.65:43395 (local address 127.15.82.126:39153)
0929 10:33:03.323211 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:03.323216 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:03.324038 (+ 822us) server_negotiation.cc:407] Connection header received
0929 10:33:03.325239 (+ 1201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:03.325242 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:03.325289 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:03.325360 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:03.326527 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:03.327048 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:03.328162 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:03.328456 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:03.328557 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:03.331444 (+ 2887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:03.331447 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:03.331449 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:03.331470 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:03.332934 (+ 1464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:03.333531 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:03.333534 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:03.333536 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:03.333619 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:03.334004 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:03.334007 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:03.334008 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:03.334195 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:03.334274 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:03.334931 (+ 657us) server_negotiation.cc:299] Negotiation successful
0929 10:33:03.335033 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260929 10:33:03.336411 15934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:03.323461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:39153 (local address 127.15.82.65:43395)
0929 10:33:03.323889 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:03.323929 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:03.325038 (+ 1109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:03.325521 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:03.325528 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:03.325860 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:03.326326 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:03.326339 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:03.327171 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:03.327174 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:03.328041 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:03.328050 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:03.328591 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:03.328593 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:03.328727 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:03.329357 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:03.329377 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:03.331276 (+ 1899us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:03.333067 (+ 1791us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:03.333073 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:03.333087 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:03.333425 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:03.333711 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:03.333715 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:03.333717 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:03.333900 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:03.334301 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:03.334311 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:03.334664 (+ 353us) client_negotiation.cc:769] Sending connection context
0929 10:33:03.334876 (+ 212us) client_negotiation.cc:240] Negotiation successful
0929 10:33:03.335241 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":100,"threads_started":1}
I20260929 10:33:03.337692 15932 heartbeater.cc:347] Connected to a master server at 127.15.82.126:39153
I20260929 10:33:03.337946 15932 heartbeater.cc:464] Registering TS with master...
I20260929 10:33:03.338557 15932 heartbeater.cc:511] Master 127.15.82.126:39153 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.15.82.66@KRBTEST.COM; defaulting to no policy
I20260929 10:33:03.340234 15741 ts_manager.cc:194] Registered new tserver with Master: 32ed1c1730974c46a546d13973f8be84 (127.15.82.65:36941)
I20260929 10:33:03.341377 15741 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.82.65@KRBTEST.COM'} at 127.15.82.65:43395
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab.
Entry for principal kudu/127.15.82.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab.
I20260929 10:33:03.357342 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.66:0
--local_ip_for_outbound_sockets=127.15.82.66
--webserver_interface=127.15.82.66
--webserver_port=0
--tserver_master_addrs=127.15.82.126:39153
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab
--principal=kudu/127.15.82.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:46749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:33:03.469000 15937 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:33:03.469280 15937 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:33:03.469379 15937 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:33:03.473553 15937 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:33:03.473613 15937 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:33:03.473711 15937 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.66
I20260929 10:33:03.479444 15937 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:46749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab
--principal=kudu/127.15.82.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.82.66
--webserver_port=0
--tserver_master_addrs=127.15.82.126:39153
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15937
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.66
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:33:03.480535 15937 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:33:03.481572 15937 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:33:03.491531 15943 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:03.491531 15945 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:03.491940 15937 server_base.cc:1034] running on GCE node
W20260929 10:33:03.491653 15942 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:03.492349 15937 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:33:03.492985 15937 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:33:03.494151 15937 hybrid_clock.cc:648] HybridClock initialized: now 1790677983494134 us; error 28 us; skew 500 ppm
Sep 29 10:33:03 dist-test-slave-vg89 krb5kdc[15696](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790677983, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:33:03.497040 15937 init.cc:377] Logged in from keytab as kudu/127.15.82.66@KRBTEST.COM (short username kudu)
I20260929 10:33:03.497903 15937 webserver.cc:460] Webserver started at http://127.15.82.66:33425/ using document root <none> and password file <none>
I20260929 10:33:03.498512 15937 fs_manager.cc:359] Metadata directory not provided
I20260929 10:33:03.498585 15937 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:33:03.498755 15937 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:33:03.500824 15937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/minicluster-data/ts-1/data/instance:
uuid: "ab33a072c6704a12ad5b1d047a09afe8"
format_stamp: "Formatted at 2026-09-29 10:33:03 on dist-test-slave-vg89"
server_key: "6327ac81fd103c7285c9955560afdac5"
server_key_iv: "50eb0db0312de9b0cabde79719a57c31"
server_key_version: "encryptionkey@0"
I20260929 10:33:03.501330 15937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/minicluster-data/ts-1/wal/instance:
uuid: "ab33a072c6704a12ad5b1d047a09afe8"
format_stamp: "Formatted at 2026-09-29 10:33:03 on dist-test-slave-vg89"
server_key: "6327ac81fd103c7285c9955560afdac5"
server_key_iv: "50eb0db0312de9b0cabde79719a57c31"
server_key_version: "encryptionkey@0"
I20260929 10:33:03.505039 15937 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260929 10:33:03.507351 15952 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:03.508669 15937 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 10:33:03.508785 15937 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/minicluster-data/ts-1/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/minicluster-data/ts-1/wal
uuid: "ab33a072c6704a12ad5b1d047a09afe8"
format_stamp: "Formatted at 2026-09-29 10:33:03 on dist-test-slave-vg89"
server_key: "6327ac81fd103c7285c9955560afdac5"
server_key_iv: "50eb0db0312de9b0cabde79719a57c31"
server_key_version: "encryptionkey@0"
I20260929 10:33:03.508888 15937 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:33:03.518189 15937 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:33:03.521541 15937 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:33:03.521706 15937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:33:03.522495 15937 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:33:03.523346 15937 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:33:03.523393 15937 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:03.523434 15937 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:33:03.523471 15937 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:03.532792 15937 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.66:40563
I20260929 10:33:03.532811 16065 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.66:40563 every 8 connection(s)
I20260929 10:33:03.533682 15937 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
Sep 29 10:33:03 dist-test-slave-vg89 krb5kdc[15696](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790677983, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.66@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:33:03.543692 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 15937
I20260929 10:33:03.543819 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/minicluster-data/ts-1/wal/instance
I20260929 10:33:03.544154 15689 external_mini_cluster.cc:1428] Setting key 490d86abd73a1658afe3bf7f4a85f0ef
I20260929 10:33:03.547858 15780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:03.535682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:52713 (local address 127.15.82.126:39153)
0929 10:33:03.535832 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:03.535835 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:03.536580 (+ 745us) server_negotiation.cc:407] Connection header received
0929 10:33:03.537884 (+ 1304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:03.537887 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:03.537943 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:03.538050 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:03.539390 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:03.539939 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:03.541154 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:03.541377 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:03.541488 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:03.544105 (+ 2617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:03.544110 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:03.544113 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:03.544142 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:03.545647 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:03.546144 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:03.546151 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:03.546156 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:03.546248 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:03.546629 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:03.546635 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:03.546640 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:03.546884 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:03.546986 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:03.547488 (+ 502us) server_negotiation.cc:299] Negotiation successful
0929 10:33:03.547642 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260929 10:33:03.548651 16068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:03.535969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:39153 (local address 127.15.82.66:52713)
0929 10:33:03.536426 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:03.536469 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:03.537668 (+ 1199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:03.538226 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:03.538233 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:03.538690 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:03.539247 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:03.539261 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:03.540091 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:03.540094 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:03.541011 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:03.541020 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:03.541540 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:03.541544 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:03.541715 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:03.542343 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:03.542359 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:03.543930 (+ 1571us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:03.545765 (+ 1835us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:03.545770 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:03.545781 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:03.546027 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:03.546347 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:03.546349 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:03.546351 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:03.546533 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:03.546982 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:03.546987 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:03.547217 (+ 230us) client_negotiation.cc:769] Sending connection context
0929 10:33:03.547387 (+ 170us) client_negotiation.cc:240] Negotiation successful
0929 10:33:03.547714 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":99,"threads_started":1}
I20260929 10:33:03.549927 16066 heartbeater.cc:347] Connected to a master server at 127.15.82.126:39153
I20260929 10:33:03.550184 16066 heartbeater.cc:464] Registering TS with master...
I20260929 10:33:03.550756 16066 heartbeater.cc:511] Master 127.15.82.126:39153 requested a full tablet report, sending...
I20260929 10:33:03.551858 15741 ts_manager.cc:194] Registered new tserver with Master: ab33a072c6704a12ad5b1d047a09afe8 (127.15.82.66:40563)
I20260929 10:33:03.552603 15741 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.82.66@KRBTEST.COM'} at 127.15.82.66:52713
WARNING: no policy specified for kudu/127.15.82.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab.
Entry for principal kudu/127.15.82.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab.
I20260929 10:33:03.577762 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.67:0
--local_ip_for_outbound_sockets=127.15.82.67
--webserver_interface=127.15.82.67
--webserver_port=0
--tserver_master_addrs=127.15.82.126:39153
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab
--principal=kudu/127.15.82.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:46749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:33:03.690642 16071 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:33:03.690907 16071 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:33:03.690977 16071 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:33:03.694960 16071 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:33:03.695031 16071 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:33:03.695142 16071 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.67
I20260929 10:33:03.700376 16071 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:46749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab
--principal=kudu/127.15.82.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.82.67
--webserver_port=0
--tserver_master_addrs=127.15.82.126:39153
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16071
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.67
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:33:03.701443 16071 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:33:03.702527 16071 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:33:03.712194 16076 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:03.712194 16079 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:03.712317 16077 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:03.712498 16071 server_base.cc:1034] running on GCE node
I20260929 10:33:03.713094 16071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:33:03.713744 16071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:33:03.714901 16071 hybrid_clock.cc:648] HybridClock initialized: now 1790677983714873 us; error 42 us; skew 500 ppm
Sep 29 10:33:03 dist-test-slave-vg89 krb5kdc[15696](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790677983, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:33:03.717558 16071 init.cc:377] Logged in from keytab as kudu/127.15.82.67@KRBTEST.COM (short username kudu)
I20260929 10:33:03.718532 16071 webserver.cc:460] Webserver started at http://127.15.82.67:37975/ using document root <none> and password file <none>
I20260929 10:33:03.719098 16071 fs_manager.cc:359] Metadata directory not provided
I20260929 10:33:03.719156 16071 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:33:03.719383 16071 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:33:03.721457 16071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/minicluster-data/ts-2/data/instance:
uuid: "e6b81b25531245b8a4263eeafda530a2"
format_stamp: "Formatted at 2026-09-29 10:33:03 on dist-test-slave-vg89"
server_key: "eecb83a5429f7a931002281d1f16b314"
server_key_iv: "5a1b85bb5049b4f737d427d6f8428d76"
server_key_version: "encryptionkey@0"
I20260929 10:33:03.722009 16071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/minicluster-data/ts-2/wal/instance:
uuid: "e6b81b25531245b8a4263eeafda530a2"
format_stamp: "Formatted at 2026-09-29 10:33:03 on dist-test-slave-vg89"
server_key: "eecb83a5429f7a931002281d1f16b314"
server_key_iv: "5a1b85bb5049b4f737d427d6f8428d76"
server_key_version: "encryptionkey@0"
I20260929 10:33:03.725692 16071 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260929 10:33:03.728092 16086 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:03.729305 16071 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260929 10:33:03.729436 16071 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/minicluster-data/ts-2/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/minicluster-data/ts-2/wal
uuid: "e6b81b25531245b8a4263eeafda530a2"
format_stamp: "Formatted at 2026-09-29 10:33:03 on dist-test-slave-vg89"
server_key: "eecb83a5429f7a931002281d1f16b314"
server_key_iv: "5a1b85bb5049b4f737d427d6f8428d76"
server_key_version: "encryptionkey@0"
I20260929 10:33:03.729548 16071 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:33:03.745712 16071 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:33:03.748852 16071 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:33:03.749044 16071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:33:03.749632 16071 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:33:03.750515 16071 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:33:03.750583 16071 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:03.750653 16071 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:33:03.750695 16071 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:03.759975 16071 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.67:37081
I20260929 10:33:03.760017 16199 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.67:37081 every 8 connection(s)
I20260929 10:33:03.760883 16071 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
I20260929 10:33:03.764335 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 16071
I20260929 10:33:03.764438 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790677981592896-15689-0/minicluster-data/ts-2/wal/instance
I20260929 10:33:03.764720 15689 external_mini_cluster.cc:1428] Setting key c4e1a98f68b550b93a280237353c993e
Sep 29 10:33:03 dist-test-slave-vg89 krb5kdc[15696](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790677983, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.67@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:33:03.774309 15780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:03.762678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.67:47567 (local address 127.15.82.126:39153)
0929 10:33:03.762845 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:03.762850 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:03.763698 (+ 848us) server_negotiation.cc:407] Connection header received
0929 10:33:03.765119 (+ 1421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:03.765122 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:03.765185 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:03.765296 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:03.766647 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:03.767257 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:03.768397 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:03.768611 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:03.768688 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:03.771128 (+ 2440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:03.771131 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:03.771133 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:03.771154 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:03.772473 (+ 1319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:03.772970 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:03.772972 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:03.772974 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:03.773045 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:03.773395 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:03.773397 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:03.773398 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:03.773581 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:03.773655 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:03.774098 (+ 443us) server_negotiation.cc:299] Negotiation successful
0929 10:33:03.774190 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260929 10:33:03.775182 16202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:03.763081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:39153 (local address 127.15.82.67:47567)
0929 10:33:03.763533 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:03.763588 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:03.764893 (+ 1305us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:03.765492 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:03.765502 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:03.765904 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:03.766510 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:03.766532 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:03.767395 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:03.767400 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:03.768289 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:03.768297 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:03.768739 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:03.768741 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:03.768878 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:03.769492 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:03.769508 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:03.770987 (+ 1479us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:03.772579 (+ 1592us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:03.772584 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:03.772596 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:03.772844 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:03.773124 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:03.773126 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:03.773127 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:03.773312 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:03.773668 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:03.773672 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:03.773895 (+ 223us) client_negotiation.cc:769] Sending connection context
0929 10:33:03.774050 (+ 155us) client_negotiation.cc:240] Negotiation successful
0929 10:33:03.774294 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":115,"threads_started":1}
I20260929 10:33:03.776238 16200 heartbeater.cc:347] Connected to a master server at 127.15.82.126:39153
I20260929 10:33:03.776479 16200 heartbeater.cc:464] Registering TS with master...
I20260929 10:33:03.776964 16200 heartbeater.cc:511] Master 127.15.82.126:39153 requested a full tablet report, sending...
I20260929 10:33:03.778015 15741 ts_manager.cc:194] Registered new tserver with Master: e6b81b25531245b8a4263eeafda530a2 (127.15.82.67:37081)
I20260929 10:33:03.778728 15741 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.82.67@KRBTEST.COM'} at 127.15.82.67:47567
I20260929 10:33:03.779332 15689 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 29 10:33:03 dist-test-slave-vg89 krb5kdc[15696](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790677983, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 29 10:33:03 dist-test-slave-vg89 krb5kdc[15696](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790677983, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.82.65@KRBTEST.COM
I20260929 10:33:03.805148 16207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:03.795952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59166 (local address 127.15.82.65:36941)
0929 10:33:03.796213 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:03.796230 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:03.796287 (+ 57us) server_negotiation.cc:407] Connection header received
0929 10:33:03.796470 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:03.796477 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:03.796672 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:03.796821 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:03.797519 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:03.798288 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:03.799234 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:03.799610 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:03.799731 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:03.802092 (+ 2361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:03.802101 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:03.802114 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:03.802138 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:03.803630 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:03.804002 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:03.804007 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:03.804009 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:03.804095 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:03.804315 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:03.804317 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:03.804318 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:03.804557 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:03.804710 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:03.804860 (+ 150us) server_negotiation.cc:299] Negotiation successful
0929 10:33:03.804981 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":67,"threads_started":1}
W20260929 10:33:03.806195 15846 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59166
Sep 29 10:33:03 dist-test-slave-vg89 krb5kdc[15696](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790677983, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 29 10:33:03 dist-test-slave-vg89 krb5kdc[15696](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790677983, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.82.65@KRBTEST.COM
I20260929 10:33:03.831488 16207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:03.823854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59168 (local address 127.15.82.65:36941)
0929 10:33:03.824026 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:03.824030 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:03.824088 (+ 58us) server_negotiation.cc:407] Connection header received
0929 10:33:03.824204 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:03.824206 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:03.824249 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:03.824319 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:03.824968 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:03.825466 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:03.826246 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:03.826525 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:03.826614 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:03.828884 (+ 2270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:03.828887 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:03.828889 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:03.828910 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:03.830154 (+ 1244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:03.830587 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:03.830590 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:03.830591 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:03.830665 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:03.830870 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:03.830872 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:03.830873 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:03.831011 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:03.831112 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:03.831272 (+ 160us) server_negotiation.cc:299] Negotiation successful
0929 10:33:03.831365 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260929 10:33:03.832543 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 15803
I20260929 10:33:03.838500 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 15937
I20260929 10:33:03.843755 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 16071
I20260929 10:33:03.849283 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 15709
2026-09-29T10:33:03Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2246 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:33:03 dist-test-slave-vg89 krb5kdc[16214](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:33:03 dist-test-slave-vg89 krb5kdc[16214](info): set up 2 sockets
Sep 29 10:33:03 dist-test-slave-vg89 krb5kdc[16214](info): commencing operation
krb5kdc: starting...
W20260929 10:33:04.890324 15689 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Sep 29 10:33:04 dist-test-slave-vg89 krb5kdc[16214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790677984, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:33:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:33:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.82.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab.
Entry for principal kudu/127.15.82.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab.
I20260929 10:33:05.013360 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:41933
--webserver_interface=127.15.82.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab
--principal=kudu/127.15.82.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:35335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.82.126:41933
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:33:05.127617 16227 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:33:05.127944 16227 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:33:05.128021 16227 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:33:05.132061 16227 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:33:05.132139 16227 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:33:05.132177 16227 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:33:05.132238 16227 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:33:05.132273 16227 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:33:05.137513 16227 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:35335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.82.126:41933
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab
--principal=kudu/127.15.82.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:41933
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.82.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16227
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:33:05.138702 16227 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:33:05.139765 16227 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 10:33:05.148443 16227 server_base.cc:1034] running on GCE node
W20260929 10:33:05.148396 16232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:05.148393 16235 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:05.148389 16233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:05.149327 16227 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:33:05.150377 16227 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:33:05.151602 16227 hybrid_clock.cc:648] HybridClock initialized: now 1790677985151572 us; error 42 us; skew 500 ppm
Sep 29 10:33:05 dist-test-slave-vg89 krb5kdc[16214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790677985, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:33:05.154373 16227 init.cc:377] Logged in from keytab as kudu/127.15.82.126@KRBTEST.COM (short username kudu)
I20260929 10:33:05.155279 16227 webserver.cc:460] Webserver started at http://127.15.82.126:36619/ using document root <none> and password file <none>
I20260929 10:33:05.155846 16227 fs_manager.cc:359] Metadata directory not provided
I20260929 10:33:05.155939 16227 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:33:05.156183 16227 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:33:05.158209 16227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/minicluster-data/master-0/data/instance:
uuid: "12d0beea1a4a4949b74f1069c1c68251"
format_stamp: "Formatted at 2026-09-29 10:33:05 on dist-test-slave-vg89"
server_key: "fac3e0e8828ea4474b8248e23f3cea38"
server_key_iv: "c24effda5f95dde02c0c41032371126d"
server_key_version: "encryptionkey@0"
I20260929 10:33:05.158845 16227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/minicluster-data/master-0/wal/instance:
uuid: "12d0beea1a4a4949b74f1069c1c68251"
format_stamp: "Formatted at 2026-09-29 10:33:05 on dist-test-slave-vg89"
server_key: "fac3e0e8828ea4474b8248e23f3cea38"
server_key_iv: "c24effda5f95dde02c0c41032371126d"
server_key_version: "encryptionkey@0"
I20260929 10:33:05.162643 16227 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260929 10:33:05.165012 16242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:05.166198 16227 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 10:33:05.166344 16227 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/minicluster-data/master-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/minicluster-data/master-0/wal
uuid: "12d0beea1a4a4949b74f1069c1c68251"
format_stamp: "Formatted at 2026-09-29 10:33:05 on dist-test-slave-vg89"
server_key: "fac3e0e8828ea4474b8248e23f3cea38"
server_key_iv: "c24effda5f95dde02c0c41032371126d"
server_key_version: "encryptionkey@0"
I20260929 10:33:05.166530 16227 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:33:05.185786 16227 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:33:05.189982 16227 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:33:05.190196 16227 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:33:05.196929 16227 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.126:41933
I20260929 10:33:05.197000 16294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.126:41933 every 8 connection(s)
I20260929 10:33:05.197891 16227 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
I20260929 10:33:05.200163 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 16227
I20260929 10:33:05.200294 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/minicluster-data/master-0/wal/instance
I20260929 10:33:05.200560 15689 external_mini_cluster.cc:1428] Setting key d0e9cac2a8a48e6d61a862c81516c012
I20260929 10:33:05.200810 16295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:33:05.207993 16295 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12d0beea1a4a4949b74f1069c1c68251: Bootstrap starting.
Sep 29 10:33:05 dist-test-slave-vg89 krb5kdc[16214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790677984, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:33:05.210474 16295 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 12d0beea1a4a4949b74f1069c1c68251: Neither blocks nor log segments found. Creating new log.
I20260929 10:33:05.211385 16295 log.cc:824] T 00000000000000000000000000000000 P 12d0beea1a4a4949b74f1069c1c68251: Log is configured to *not* fsync() on all Append() calls
I20260929 10:33:05.213765 16295 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12d0beea1a4a4949b74f1069c1c68251: No bootstrap required, opened a new log
I20260929 10:33:05.215749 16298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:05.201965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35428 (local address 127.15.82.126:41933)
0929 10:33:05.202378 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:05.202428 (+ 50us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:05.202473 (+ 45us) server_negotiation.cc:407] Connection header received
0929 10:33:05.203516 (+ 1043us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:05.203541 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:05.203963 (+ 422us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:05.204331 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:05.204982 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:05.206478 (+ 1496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:05.207298 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:05.207977 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:05.208147 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:05.210871 (+ 2724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:05.210878 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:05.210894 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:05.210926 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:05.212788 (+ 1862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:05.213203 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:05.213209 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:05.213214 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:05.213349 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:05.213599 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:05.213602 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:05.213604 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:05.214102 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:05.214289 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:05.214518 (+ 229us) server_negotiation.cc:299] Negotiation successful
0929 10:33:05.214820 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":114,"threads_started":1}
I20260929 10:33:05.218102 16295 raft_consensus.cc:348] T 00000000000000000000000000000000 P 12d0beea1a4a4949b74f1069c1c68251 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12d0beea1a4a4949b74f1069c1c68251" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 41933 } }
I20260929 10:33:05.218310 16295 raft_consensus.cc:374] T 00000000000000000000000000000000 P 12d0beea1a4a4949b74f1069c1c68251 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:33:05.218344 16295 raft_consensus.cc:729] T 00000000000000000000000000000000 P 12d0beea1a4a4949b74f1069c1c68251 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12d0beea1a4a4949b74f1069c1c68251, State: Initialized, Role: FOLLOWER
I20260929 10:33:05.218798 16295 consensus_queue.cc:260] T 00000000000000000000000000000000 P 12d0beea1a4a4949b74f1069c1c68251 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12d0beea1a4a4949b74f1069c1c68251" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 41933 } }
I20260929 10:33:05.218920 16295 raft_consensus.cc:388] T 00000000000000000000000000000000 P 12d0beea1a4a4949b74f1069c1c68251 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:33:05.218964 16295 raft_consensus.cc:482] T 00000000000000000000000000000000 P 12d0beea1a4a4949b74f1069c1c68251 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:33:05.219033 16295 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 12d0beea1a4a4949b74f1069c1c68251 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:33:05.219822 16295 raft_consensus.cc:504] T 00000000000000000000000000000000 P 12d0beea1a4a4949b74f1069c1c68251 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12d0beea1a4a4949b74f1069c1c68251" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 41933 } }
I20260929 10:33:05.220108 16295 leader_election.cc:302] T 00000000000000000000000000000000 P 12d0beea1a4a4949b74f1069c1c68251 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 12d0beea1a4a4949b74f1069c1c68251; no voters:
I20260929 10:33:05.220376 16295 leader_election.cc:288] T 00000000000000000000000000000000 P 12d0beea1a4a4949b74f1069c1c68251 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:33:05.220494 16300 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 12d0beea1a4a4949b74f1069c1c68251 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:33:05.220750 16300 raft_consensus.cc:686] T 00000000000000000000000000000000 P 12d0beea1a4a4949b74f1069c1c68251 [term 1 LEADER]: Becoming Leader. State: Replica: 12d0beea1a4a4949b74f1069c1c68251, State: Running, Role: LEADER
I20260929 10:33:05.221045 16300 consensus_queue.cc:237] T 00000000000000000000000000000000 P 12d0beea1a4a4949b74f1069c1c68251 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12d0beea1a4a4949b74f1069c1c68251" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 41933 } }
I20260929 10:33:05.221621 16295 sys_catalog.cc:564] T 00000000000000000000000000000000 P 12d0beea1a4a4949b74f1069c1c68251 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:33:05.222723 16302 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12d0beea1a4a4949b74f1069c1c68251 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 12d0beea1a4a4949b74f1069c1c68251. Latest consensus state: current_term: 1 leader_uuid: "12d0beea1a4a4949b74f1069c1c68251" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12d0beea1a4a4949b74f1069c1c68251" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 41933 } } }
I20260929 10:33:05.222859 16302 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12d0beea1a4a4949b74f1069c1c68251 [sys.catalog]: This master's current role is: LEADER
I20260929 10:33:05.222714 16301 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12d0beea1a4a4949b74f1069c1c68251 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "12d0beea1a4a4949b74f1069c1c68251" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12d0beea1a4a4949b74f1069c1c68251" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 41933 } } }
I20260929 10:33:05.223140 16301 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12d0beea1a4a4949b74f1069c1c68251 [sys.catalog]: This master's current role is: LEADER
I20260929 10:33:05.223366 16309 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:33:05.226444 16309 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:33:05.232110 16309 catalog_manager.cc:1347] Generated new cluster ID: 19d06ce294d94b2aaa580d4d98613ac3
I20260929 10:33:05.232183 16309 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:33:05.240046 16309 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:33:05.240939 16309 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:33:05.248979 16309 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 12d0beea1a4a4949b74f1069c1c68251: Generated new TSK 0
I20260929 10:33:05.249624 16309 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.82.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab.
Entry for principal kudu/127.15.82.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab.
I20260929 10:33:05.283056 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.65:0
--local_ip_for_outbound_sockets=127.15.82.65
--webserver_interface=127.15.82.65
--webserver_port=0
--tserver_master_addrs=127.15.82.126:41933
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab
--principal=kudu/127.15.82.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:35335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:33:05.394064 16321 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:33:05.394352 16321 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:33:05.394433 16321 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:33:05.398597 16321 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:33:05.398658 16321 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:33:05.398720 16321 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.65
I20260929 10:33:05.403968 16321 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:35335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab
--principal=kudu/127.15.82.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.82.65
--webserver_port=0
--tserver_master_addrs=127.15.82.126:41933
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16321
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.65
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:33:05.404974 16321 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:33:05.405956 16321 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:33:05.415331 16327 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:05.415416 16329 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:05.415473 16326 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:05.415726 16321 server_base.cc:1034] running on GCE node
I20260929 10:33:05.416113 16321 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:33:05.416741 16321 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:33:05.417928 16321 hybrid_clock.cc:648] HybridClock initialized: now 1790677985417913 us; error 26 us; skew 500 ppm
Sep 29 10:33:05 dist-test-slave-vg89 krb5kdc[16214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790677985, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:33:05.420756 16321 init.cc:377] Logged in from keytab as kudu/127.15.82.65@KRBTEST.COM (short username kudu)
I20260929 10:33:05.421645 16321 webserver.cc:460] Webserver started at http://127.15.82.65:36345/ using document root <none> and password file <none>
I20260929 10:33:05.422304 16321 fs_manager.cc:359] Metadata directory not provided
I20260929 10:33:05.422428 16321 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:33:05.422650 16321 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:33:05.424715 16321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/minicluster-data/ts-0/data/instance:
uuid: "752e3a6dba8942d4b3608e79bd812dd4"
format_stamp: "Formatted at 2026-09-29 10:33:05 on dist-test-slave-vg89"
server_key: "835e8a4de9662565a7296a4c0a2e1093"
server_key_iv: "2cb0bca251fd892cb5fecc9760d9a775"
server_key_version: "encryptionkey@0"
I20260929 10:33:05.425341 16321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/minicluster-data/ts-0/wal/instance:
uuid: "752e3a6dba8942d4b3608e79bd812dd4"
format_stamp: "Formatted at 2026-09-29 10:33:05 on dist-test-slave-vg89"
server_key: "835e8a4de9662565a7296a4c0a2e1093"
server_key_iv: "2cb0bca251fd892cb5fecc9760d9a775"
server_key_version: "encryptionkey@0"
I20260929 10:33:05.428990 16321 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.006s
I20260929 10:33:05.431396 16336 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:05.432597 16321 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 10:33:05.432727 16321 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/minicluster-data/ts-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/minicluster-data/ts-0/wal
uuid: "752e3a6dba8942d4b3608e79bd812dd4"
format_stamp: "Formatted at 2026-09-29 10:33:05 on dist-test-slave-vg89"
server_key: "835e8a4de9662565a7296a4c0a2e1093"
server_key_iv: "2cb0bca251fd892cb5fecc9760d9a775"
server_key_version: "encryptionkey@0"
I20260929 10:33:05.432835 16321 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:33:05.447788 16321 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:33:05.451145 16321 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:33:05.451340 16321 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:33:05.451920 16321 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:33:05.452715 16321 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:33:05.452781 16321 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:05.452852 16321 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:33:05.452903 16321 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:05.461539 16321 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.65:38803
I20260929 10:33:05.461555 16449 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.65:38803 every 8 connection(s)
I20260929 10:33:05.462462 16321 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
I20260929 10:33:05.469271 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 16321
I20260929 10:33:05.469378 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/minicluster-data/ts-0/wal/instance
I20260929 10:33:05.469669 15689 external_mini_cluster.cc:1428] Setting key a974a067c34c0f4f8d03406620043ab9
Sep 29 10:33:05 dist-test-slave-vg89 krb5kdc[16214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790677985, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.65@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:33:05.476850 16298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:05.464298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.65:54469 (local address 127.15.82.126:41933)
0929 10:33:05.464438 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:05.464443 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:05.465347 (+ 904us) server_negotiation.cc:407] Connection header received
0929 10:33:05.466667 (+ 1320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:05.466670 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:05.466727 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:05.466804 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:05.467929 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:05.468476 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:05.469936 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:05.470271 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:05.470373 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:05.473098 (+ 2725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:05.473101 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:05.473103 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:05.473125 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:05.474790 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:05.475310 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:05.475317 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:05.475322 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:05.475423 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:05.475751 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:05.475755 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:05.475756 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:05.475938 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:05.476049 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:05.476529 (+ 480us) server_negotiation.cc:299] Negotiation successful
0929 10:33:05.476671 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260929 10:33:05.477721 16452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:05.464714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:41933 (local address 127.15.82.65:54469)
0929 10:33:05.465204 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:05.465247 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:05.466457 (+ 1210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:05.466974 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:05.466981 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:05.467318 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:05.467784 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:05.467797 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:05.468608 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:05.468616 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:05.469817 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:05.469828 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:05.470461 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:05.470468 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:05.470685 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:05.471339 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:05.471359 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:05.472907 (+ 1548us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:05.474900 (+ 1993us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:05.474917 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:05.474928 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:05.475207 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:05.475518 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:05.475520 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:05.475521 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:05.475650 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:05.476046 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:05.476066 (+ 20us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:05.476299 (+ 233us) client_negotiation.cc:769] Sending connection context
0929 10:33:05.476487 (+ 188us) client_negotiation.cc:240] Negotiation successful
0929 10:33:05.476816 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":101,"threads_started":1}
I20260929 10:33:05.478999 16450 heartbeater.cc:347] Connected to a master server at 127.15.82.126:41933
I20260929 10:33:05.479262 16450 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.15.82.66@KRBTEST.COM; defaulting to no policy
I20260929 10:33:05.479840 16450 heartbeater.cc:511] Master 127.15.82.126:41933 requested a full tablet report, sending...
I20260929 10:33:05.481335 16252 ts_manager.cc:194] Registered new tserver with Master: 752e3a6dba8942d4b3608e79bd812dd4 (127.15.82.65:38803)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.66@KRBTEST.COM" created.
I20260929 10:33:05.482559 16252 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.82.65@KRBTEST.COM'} at 127.15.82.65:54469
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab.
Entry for principal kudu/127.15.82.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab.
I20260929 10:33:05.497267 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.66:0
--local_ip_for_outbound_sockets=127.15.82.66
--webserver_interface=127.15.82.66
--webserver_port=0
--tserver_master_addrs=127.15.82.126:41933
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab
--principal=kudu/127.15.82.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:35335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:33:05.609164 16455 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:33:05.609424 16455 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:33:05.609468 16455 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:33:05.613476 16455 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:33:05.613531 16455 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:33:05.613590 16455 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.66
I20260929 10:33:05.618816 16455 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:35335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab
--principal=kudu/127.15.82.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.82.66
--webserver_port=0
--tserver_master_addrs=127.15.82.126:41933
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16455
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.66
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:33:05.619794 16455 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:33:05.620801 16455 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:33:05.630647 16460 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:05.630647 16463 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:05.630641 16461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:05.630815 16455 server_base.cc:1034] running on GCE node
I20260929 10:33:05.631294 16455 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:33:05.631947 16455 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:33:05.633124 16455 hybrid_clock.cc:648] HybridClock initialized: now 1790677985633109 us; error 28 us; skew 500 ppm
Sep 29 10:33:05 dist-test-slave-vg89 krb5kdc[16214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790677985, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:33:05.635848 16455 init.cc:377] Logged in from keytab as kudu/127.15.82.66@KRBTEST.COM (short username kudu)
I20260929 10:33:05.636817 16455 webserver.cc:460] Webserver started at http://127.15.82.66:34601/ using document root <none> and password file <none>
I20260929 10:33:05.637384 16455 fs_manager.cc:359] Metadata directory not provided
I20260929 10:33:05.637472 16455 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:33:05.637712 16455 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:33:05.639822 16455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/minicluster-data/ts-1/data/instance:
uuid: "d50490450e084a928b6a6e2fd376c87e"
format_stamp: "Formatted at 2026-09-29 10:33:05 on dist-test-slave-vg89"
server_key: "1e46b4edbdb3e4dfd80c14a80f8cf698"
server_key_iv: "6228bab2d09a9242bf8ee4bb60d92c29"
server_key_version: "encryptionkey@0"
I20260929 10:33:05.640399 16455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/minicluster-data/ts-1/wal/instance:
uuid: "d50490450e084a928b6a6e2fd376c87e"
format_stamp: "Formatted at 2026-09-29 10:33:05 on dist-test-slave-vg89"
server_key: "1e46b4edbdb3e4dfd80c14a80f8cf698"
server_key_iv: "6228bab2d09a9242bf8ee4bb60d92c29"
server_key_version: "encryptionkey@0"
I20260929 10:33:05.644125 16455 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260929 10:33:05.646478 16470 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:05.647701 16455 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 10:33:05.647831 16455 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/minicluster-data/ts-1/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/minicluster-data/ts-1/wal
uuid: "d50490450e084a928b6a6e2fd376c87e"
format_stamp: "Formatted at 2026-09-29 10:33:05 on dist-test-slave-vg89"
server_key: "1e46b4edbdb3e4dfd80c14a80f8cf698"
server_key_iv: "6228bab2d09a9242bf8ee4bb60d92c29"
server_key_version: "encryptionkey@0"
I20260929 10:33:05.647936 16455 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:33:05.662112 16455 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:33:05.665468 16455 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:33:05.665690 16455 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:33:05.666321 16455 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:33:05.667258 16455 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:33:05.667337 16455 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:05.667410 16455 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:33:05.667443 16455 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:05.676616 16455 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.66:41811
I20260929 10:33:05.676638 16583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.66:41811 every 8 connection(s)
I20260929 10:33:05.677556 16455 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
I20260929 10:33:05.683507 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 16455
I20260929 10:33:05.683660 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/minicluster-data/ts-1/wal/instance
I20260929 10:33:05.683933 15689 external_mini_cluster.cc:1428] Setting key 346c9ec79799cef5f2263e8225a6dcb2
Sep 29 10:33:05 dist-test-slave-vg89 krb5kdc[16214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790677985, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.66@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:33:05.692631 16298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:05.679565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:57405 (local address 127.15.82.126:41933)
0929 10:33:05.679702 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:05.679707 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:05.680562 (+ 855us) server_negotiation.cc:407] Connection header received
0929 10:33:05.681863 (+ 1301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:05.681866 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:05.681927 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:05.682013 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:05.683329 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:05.683984 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:05.685483 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:05.685710 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:05.685793 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:05.688638 (+ 2845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:05.688644 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:05.688650 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:05.688684 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:05.690391 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:05.690899 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:05.690906 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:05.690909 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:05.691009 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:05.691331 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:05.691336 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:05.691340 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:05.691647 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:05.691765 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:05.692279 (+ 514us) server_negotiation.cc:299] Negotiation successful
0929 10:33:05.692413 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 10:33:05.693435 16586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:05.679933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:41933 (local address 127.15.82.66:57405)
0929 10:33:05.680405 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:05.680450 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:05.681645 (+ 1195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:05.682212 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:05.682220 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:05.682650 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:05.683157 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:05.683171 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:05.684136 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:05.684140 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:05.685365 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:05.685377 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:05.685881 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:05.685889 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:05.686116 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:05.686791 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:05.686817 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:05.688468 (+ 1651us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:05.690515 (+ 2047us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:05.690527 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:05.690540 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:05.690793 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:05.691109 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:05.691111 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:05.691112 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:05.691237 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:05.691754 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:05.691759 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:05.691995 (+ 236us) client_negotiation.cc:769] Sending connection context
0929 10:33:05.692159 (+ 164us) client_negotiation.cc:240] Negotiation successful
0929 10:33:05.692512 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":128,"threads_started":1}
WARNING: no policy specified for kudu/127.15.82.67@KRBTEST.COM; defaulting to no policy
I20260929 10:33:05.694689 16584 heartbeater.cc:347] Connected to a master server at 127.15.82.126:41933
I20260929 10:33:05.694906 16584 heartbeater.cc:464] Registering TS with master...
I20260929 10:33:05.695397 16584 heartbeater.cc:511] Master 127.15.82.126:41933 requested a full tablet report, sending...
I20260929 10:33:05.696491 16252 ts_manager.cc:194] Registered new tserver with Master: d50490450e084a928b6a6e2fd376c87e (127.15.82.66:41811)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.67@KRBTEST.COM" created.
I20260929 10:33:05.697144 16252 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.82.66@KRBTEST.COM'} at 127.15.82.66:57405
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab.
Entry for principal kudu/127.15.82.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab.
I20260929 10:33:05.713163 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.67:0
--local_ip_for_outbound_sockets=127.15.82.67
--webserver_interface=127.15.82.67
--webserver_port=0
--tserver_master_addrs=127.15.82.126:41933
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab
--principal=kudu/127.15.82.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:35335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:33:05.827971 16589 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:33:05.828277 16589 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:33:05.828354 16589 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:33:05.832479 16589 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:33:05.832571 16589 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:33:05.832664 16589 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.67
I20260929 10:33:05.838263 16589 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:35335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab
--principal=kudu/127.15.82.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.82.67
--webserver_port=0
--tserver_master_addrs=127.15.82.126:41933
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16589
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.67
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:33:05.839432 16589 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:33:05.840538 16589 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:33:05.850314 16597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:05.850380 16589 server_base.cc:1034] running on GCE node
W20260929 10:33:05.850308 16595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:05.850315 16594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:05.851039 16589 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:33:05.851797 16589 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:33:05.852979 16589 hybrid_clock.cc:648] HybridClock initialized: now 1790677985852959 us; error 32 us; skew 500 ppm
Sep 29 10:33:05 dist-test-slave-vg89 krb5kdc[16214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790677985, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:33:05.856000 16589 init.cc:377] Logged in from keytab as kudu/127.15.82.67@KRBTEST.COM (short username kudu)
I20260929 10:33:05.856827 16589 webserver.cc:460] Webserver started at http://127.15.82.67:42651/ using document root <none> and password file <none>
I20260929 10:33:05.857362 16589 fs_manager.cc:359] Metadata directory not provided
I20260929 10:33:05.857417 16589 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:33:05.857582 16589 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:33:05.859723 16589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/minicluster-data/ts-2/data/instance:
uuid: "56c48e5e70aa4d5a8df52fb119bbc087"
format_stamp: "Formatted at 2026-09-29 10:33:05 on dist-test-slave-vg89"
server_key: "114509678c0ea4254101038dc1d01123"
server_key_iv: "483d768759e9c8df7356b6961abc98a4"
server_key_version: "encryptionkey@0"
I20260929 10:33:05.860311 16589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/minicluster-data/ts-2/wal/instance:
uuid: "56c48e5e70aa4d5a8df52fb119bbc087"
format_stamp: "Formatted at 2026-09-29 10:33:05 on dist-test-slave-vg89"
server_key: "114509678c0ea4254101038dc1d01123"
server_key_iv: "483d768759e9c8df7356b6961abc98a4"
server_key_version: "encryptionkey@0"
I20260929 10:33:05.863936 16589 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260929 10:33:05.866233 16604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:05.867480 16589 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 10:33:05.867573 16589 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/minicluster-data/ts-2/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/minicluster-data/ts-2/wal
uuid: "56c48e5e70aa4d5a8df52fb119bbc087"
format_stamp: "Formatted at 2026-09-29 10:33:05 on dist-test-slave-vg89"
server_key: "114509678c0ea4254101038dc1d01123"
server_key_iv: "483d768759e9c8df7356b6961abc98a4"
server_key_version: "encryptionkey@0"
I20260929 10:33:05.867658 16589 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:33:05.882661 16589 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:33:05.886014 16589 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:33:05.886185 16589 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:33:05.886803 16589 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:33:05.887615 16589 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:33:05.887662 16589 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:05.887702 16589 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:33:05.887720 16589 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:05.897475 16589 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.67:41595
I20260929 10:33:05.897496 16717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.67:41595 every 8 connection(s)
I20260929 10:33:05.898425 16589 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
I20260929 10:33:05.900097 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 16589
I20260929 10:33:05.900240 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790677981592896-15689-0/minicluster-data/ts-2/wal/instance
I20260929 10:33:05.900552 15689 external_mini_cluster.cc:1428] Setting key 3b6f234da6248e0f6b2b29a7ebfa3b09
Sep 29 10:33:05 dist-test-slave-vg89 krb5kdc[16214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790677985, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.67@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:33:05.912339 16298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:05.900309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.67:59673 (local address 127.15.82.126:41933)
0929 10:33:05.900491 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:05.900497 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:05.901507 (+ 1010us) server_negotiation.cc:407] Connection header received
0929 10:33:05.903051 (+ 1544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:05.903054 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:05.903110 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:05.903185 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:05.904397 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:05.904946 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:05.906138 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:05.906377 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:05.906526 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:05.908966 (+ 2440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:05.908972 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:05.908979 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:05.909005 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:05.910244 (+ 1239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:05.910789 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:05.910793 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:05.910795 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:05.910882 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:05.911239 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:05.911244 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:05.911246 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:05.911433 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:05.911553 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:05.912041 (+ 488us) server_negotiation.cc:299] Negotiation successful
0929 10:33:05.912173 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260929 10:33:05.913168 16720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:05.900798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:41933 (local address 127.15.82.67:59673)
0929 10:33:05.901353 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:05.901403 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:05.902804 (+ 1401us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:05.903369 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:05.903377 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:05.903743 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:05.904242 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:05.904257 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:05.905086 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:05.905089 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:05.905993 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:05.906003 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:05.906568 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:05.906572 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:05.906757 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:05.907360 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:05.907375 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:05.908793 (+ 1418us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:05.910363 (+ 1570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:05.910369 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:05.910386 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:05.910680 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:05.910986 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:05.910989 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:05.910991 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:05.911140 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:05.911542 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:05.911549 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:05.911801 (+ 252us) client_negotiation.cc:769] Sending connection context
0929 10:33:05.912000 (+ 199us) client_negotiation.cc:240] Negotiation successful
0929 10:33:05.912339 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":127,"threads_started":1}
I20260929 10:33:05.914466 16718 heartbeater.cc:347] Connected to a master server at 127.15.82.126:41933
I20260929 10:33:05.914729 16718 heartbeater.cc:464] Registering TS with master...
I20260929 10:33:05.915254 16718 heartbeater.cc:511] Master 127.15.82.126:41933 requested a full tablet report, sending...
I20260929 10:33:05.916257 16259 ts_manager.cc:194] Registered new tserver with Master: 56c48e5e70aa4d5a8df52fb119bbc087 (127.15.82.67:41595)
I20260929 10:33:05.916800 16259 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.82.67@KRBTEST.COM'} at 127.15.82.67:59673
I20260929 10:33:05.926721 15689 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 10:33:05.937077 16298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:05.930125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35438 (local address 127.15.82.126:41933)
0929 10:33:05.930232 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:05.930235 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:05.930486 (+ 251us) server_negotiation.cc:407] Connection header received
0929 10:33:05.930602 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:05.930605 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:05.930643 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:05.930712 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:05.931461 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:05.931978 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:05.932750 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:05.932956 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:05.933031 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:05.934283 (+ 1252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:05.934286 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:05.934288 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:05.934311 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:05.935619 (+ 1308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:05.935988 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:05.935991 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:05.935992 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:05.936064 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:05.936282 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:05.936284 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:05.936285 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:05.936542 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:05.936656 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:05.936857 (+ 201us) server_negotiation.cc:299] Negotiation successful
0929 10:33:05.936955 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260929 10:33:05.941956 16259 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35438:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 10:33:05.945143 16259 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:33:05.959638 16733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:05.953485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.66:41811 (local address 127.0.0.1:48202)
0929 10:33:05.954849 (+ 1364us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:05.954863 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:05.954969 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:05.955486 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:05.955495 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:05.955522 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:33:05.955690 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:05.955701 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:05.956864 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:05.956871 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:05.958126 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:05.958135 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:05.959013 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:05.959025 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:05.959142 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:05.959182 (+ 40us) client_negotiation.cc:769] Sending connection context
0929 10:33:05.959346 (+ 164us) client_negotiation.cc:240] Negotiation successful
0929 10:33:05.959475 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1300,"mutex_wait_us":25,"spinlock_wait_cycles":9088,"thread_start_us":120,"threads_started":1}
I20260929 10:33:05.959640 16732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:05.954279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48202 (local address 127.15.82.66:41811)
0929 10:33:05.954711 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:05.954726 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:05.954964 (+ 238us) server_negotiation.cc:407] Connection header received
0929 10:33:05.955081 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:05.955088 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:05.955313 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:05.955476 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:33:05.955818 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:05.956740 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:05.958248 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:05.958889 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:05.959014 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:05.959170 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:05.959377 (+ 207us) server_negotiation.cc:299] Negotiation successful
0929 10:33:05.959475 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":136,"threads_started":1}
I20260929 10:33:05.960809 16729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:05.953471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.65:38803 (local address 127.0.0.1:42668)
0929 10:33:05.954280 (+ 809us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:05.954334 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:05.954496 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:05.955722 (+ 1226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:05.955726 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:05.955747 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:33:05.955858 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:05.955863 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:05.957391 (+ 1528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:05.957394 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:05.958924 (+ 1530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:05.958936 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:05.960252 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:05.960261 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:05.960388 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:05.960470 (+ 82us) client_negotiation.cc:769] Sending connection context
0929 10:33:05.960557 (+ 87us) client_negotiation.cc:240] Negotiation successful
0929 10:33:05.960656 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":634,"mutex_wait_us":185,"thread_start_us":71,"threads_started":1}
I20260929 10:33:05.960829 16730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:05.954350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.67:41595 (local address 127.0.0.1:53164)
0929 10:33:05.955157 (+ 807us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:05.955169 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:05.955256 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:05.955834 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:05.955837 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:05.955858 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:33:05.955962 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:05.955967 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:05.957391 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:05.957394 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:05.958931 (+ 1537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:05.958936 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:05.960041 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:05.960045 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:05.960445 (+ 400us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:05.960479 (+ 34us) client_negotiation.cc:769] Sending connection context
0929 10:33:05.960573 (+ 94us) client_negotiation.cc:240] Negotiation successful
0929 10:33:05.960680 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":746,"mutex_wait_us":198,"thread_start_us":125,"threads_started":1}
I20260929 10:33:05.961400 16734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:05.954789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53164 (local address 127.15.82.67:41595)
0929 10:33:05.955157 (+ 368us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:05.955169 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:05.955250 (+ 81us) server_negotiation.cc:407] Connection header received
0929 10:33:05.955357 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:05.955363 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:05.955534 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:05.955679 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:33:05.956098 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:05.957234 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:05.959080 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:05.959915 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:05.960025 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:05.960184 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:05.961207 (+ 1023us) server_negotiation.cc:299] Negotiation successful
0929 10:33:05.961286 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":156,"threads_started":1}
I20260929 10:33:05.960834 16731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:05.953615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42668 (local address 127.15.82.65:38803)
0929 10:33:05.953888 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:05.953912 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:05.954652 (+ 740us) server_negotiation.cc:407] Connection header received
0929 10:33:05.955176 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:05.955185 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:05.955397 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:05.955564 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:33:05.955967 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:05.957222 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:05.959073 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:05.959915 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:05.960024 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:05.960184 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:05.960602 (+ 418us) server_negotiation.cc:299] Negotiation successful
0929 10:33:05.960702 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":102,"threads_started":1}
I20260929 10:33:05.963835 16384 tablet_service.cc:1467] Processing CreateTablet for tablet 17168a1422cc4f7691a852382f021e82 (DEFAULT_TABLE table=test-table [id=be440e44b7014628b1ce02c5129f3541]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:33:05.963806 16518 tablet_service.cc:1467] Processing CreateTablet for tablet 17168a1422cc4f7691a852382f021e82 (DEFAULT_TABLE table=test-table [id=be440e44b7014628b1ce02c5129f3541]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:33:05.964211 16652 tablet_service.cc:1467] Processing CreateTablet for tablet 17168a1422cc4f7691a852382f021e82 (DEFAULT_TABLE table=test-table [id=be440e44b7014628b1ce02c5129f3541]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:33:05.964833 16384 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17168a1422cc4f7691a852382f021e82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:33:05.964846 16518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17168a1422cc4f7691a852382f021e82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:33:05.965086 16652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17168a1422cc4f7691a852382f021e82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:33:05.972533 16735 tablet_bootstrap.cc:492] T 17168a1422cc4f7691a852382f021e82 P 752e3a6dba8942d4b3608e79bd812dd4: Bootstrap starting.
I20260929 10:33:05.972533 16737 tablet_bootstrap.cc:492] T 17168a1422cc4f7691a852382f021e82 P d50490450e084a928b6a6e2fd376c87e: Bootstrap starting.
I20260929 10:33:05.974720 16736 tablet_bootstrap.cc:492] T 17168a1422cc4f7691a852382f021e82 P 56c48e5e70aa4d5a8df52fb119bbc087: Bootstrap starting.
I20260929 10:33:05.974972 16735 tablet_bootstrap.cc:654] T 17168a1422cc4f7691a852382f021e82 P 752e3a6dba8942d4b3608e79bd812dd4: Neither blocks nor log segments found. Creating new log.
I20260929 10:33:05.975481 16737 tablet_bootstrap.cc:654] T 17168a1422cc4f7691a852382f021e82 P d50490450e084a928b6a6e2fd376c87e: Neither blocks nor log segments found. Creating new log.
I20260929 10:33:05.976308 16735 log.cc:824] T 17168a1422cc4f7691a852382f021e82 P 752e3a6dba8942d4b3608e79bd812dd4: Log is configured to *not* fsync() on all Append() calls
I20260929 10:33:05.976308 16737 log.cc:824] T 17168a1422cc4f7691a852382f021e82 P d50490450e084a928b6a6e2fd376c87e: Log is configured to *not* fsync() on all Append() calls
I20260929 10:33:05.977654 16736 tablet_bootstrap.cc:654] T 17168a1422cc4f7691a852382f021e82 P 56c48e5e70aa4d5a8df52fb119bbc087: Neither blocks nor log segments found. Creating new log.
I20260929 10:33:05.978281 16735 tablet_bootstrap.cc:492] T 17168a1422cc4f7691a852382f021e82 P 752e3a6dba8942d4b3608e79bd812dd4: No bootstrap required, opened a new log
I20260929 10:33:05.978439 16737 tablet_bootstrap.cc:492] T 17168a1422cc4f7691a852382f021e82 P d50490450e084a928b6a6e2fd376c87e: No bootstrap required, opened a new log
I20260929 10:33:05.978530 16735 ts_tablet_manager.cc:1397] T 17168a1422cc4f7691a852382f021e82 P 752e3a6dba8942d4b3608e79bd812dd4: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260929 10:33:05.978503 16736 log.cc:824] T 17168a1422cc4f7691a852382f021e82 P 56c48e5e70aa4d5a8df52fb119bbc087: Log is configured to *not* fsync() on all Append() calls
I20260929 10:33:05.978616 16737 ts_tablet_manager.cc:1397] T 17168a1422cc4f7691a852382f021e82 P d50490450e084a928b6a6e2fd376c87e: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260929 10:33:05.980410 16736 tablet_bootstrap.cc:492] T 17168a1422cc4f7691a852382f021e82 P 56c48e5e70aa4d5a8df52fb119bbc087: No bootstrap required, opened a new log
I20260929 10:33:05.980602 16736 ts_tablet_manager.cc:1397] T 17168a1422cc4f7691a852382f021e82 P 56c48e5e70aa4d5a8df52fb119bbc087: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260929 10:33:05.981647 16735 raft_consensus.cc:348] T 17168a1422cc4f7691a852382f021e82 P 752e3a6dba8942d4b3608e79bd812dd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "752e3a6dba8942d4b3608e79bd812dd4" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 38803 } } peers { permanent_uuid: "d50490450e084a928b6a6e2fd376c87e" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 41811 } } peers { permanent_uuid: "56c48e5e70aa4d5a8df52fb119bbc087" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 41595 } }
I20260929 10:33:05.981889 16735 raft_consensus.cc:374] T 17168a1422cc4f7691a852382f021e82 P 752e3a6dba8942d4b3608e79bd812dd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:33:05.981949 16735 raft_consensus.cc:729] T 17168a1422cc4f7691a852382f021e82 P 752e3a6dba8942d4b3608e79bd812dd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 752e3a6dba8942d4b3608e79bd812dd4, State: Initialized, Role: FOLLOWER
I20260929 10:33:05.982450 16735 consensus_queue.cc:260] T 17168a1422cc4f7691a852382f021e82 P 752e3a6dba8942d4b3608e79bd812dd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "752e3a6dba8942d4b3608e79bd812dd4" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 38803 } } peers { permanent_uuid: "d50490450e084a928b6a6e2fd376c87e" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 41811 } } peers { permanent_uuid: "56c48e5e70aa4d5a8df52fb119bbc087" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 41595 } }
I20260929 10:33:05.982354 16737 raft_consensus.cc:348] T 17168a1422cc4f7691a852382f021e82 P d50490450e084a928b6a6e2fd376c87e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "752e3a6dba8942d4b3608e79bd812dd4" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 38803 } } peers { permanent_uuid: "d50490450e084a928b6a6e2fd376c87e" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 41811 } } peers { permanent_uuid: "56c48e5e70aa4d5a8df52fb119bbc087" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 41595 } }
I20260929 10:33:05.982625 16737 raft_consensus.cc:374] T 17168a1422cc4f7691a852382f021e82 P d50490450e084a928b6a6e2fd376c87e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:33:05.982682 16737 raft_consensus.cc:729] T 17168a1422cc4f7691a852382f021e82 P d50490450e084a928b6a6e2fd376c87e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d50490450e084a928b6a6e2fd376c87e, State: Initialized, Role: FOLLOWER
I20260929 10:33:05.983037 16450 heartbeater.cc:503] Master 127.15.82.126:41933 was elected leader, sending a full tablet report...
I20260929 10:33:05.983421 16735 ts_tablet_manager.cc:1428] T 17168a1422cc4f7691a852382f021e82 P 752e3a6dba8942d4b3608e79bd812dd4: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260929 10:33:05.983395 16737 consensus_queue.cc:260] T 17168a1422cc4f7691a852382f021e82 P d50490450e084a928b6a6e2fd376c87e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "752e3a6dba8942d4b3608e79bd812dd4" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 38803 } } peers { permanent_uuid: "d50490450e084a928b6a6e2fd376c87e" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 41811 } } peers { permanent_uuid: "56c48e5e70aa4d5a8df52fb119bbc087" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 41595 } }
I20260929 10:33:05.984126 16584 heartbeater.cc:503] Master 127.15.82.126:41933 was elected leader, sending a full tablet report...
I20260929 10:33:05.984328 16737 ts_tablet_manager.cc:1428] T 17168a1422cc4f7691a852382f021e82 P d50490450e084a928b6a6e2fd376c87e: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260929 10:33:05.984910 16736 raft_consensus.cc:348] T 17168a1422cc4f7691a852382f021e82 P 56c48e5e70aa4d5a8df52fb119bbc087 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "752e3a6dba8942d4b3608e79bd812dd4" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 38803 } } peers { permanent_uuid: "d50490450e084a928b6a6e2fd376c87e" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 41811 } } peers { permanent_uuid: "56c48e5e70aa4d5a8df52fb119bbc087" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 41595 } }
I20260929 10:33:05.985215 16736 raft_consensus.cc:374] T 17168a1422cc4f7691a852382f021e82 P 56c48e5e70aa4d5a8df52fb119bbc087 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:33:05.985278 16736 raft_consensus.cc:729] T 17168a1422cc4f7691a852382f021e82 P 56c48e5e70aa4d5a8df52fb119bbc087 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56c48e5e70aa4d5a8df52fb119bbc087, State: Initialized, Role: FOLLOWER
I20260929 10:33:05.985678 16736 consensus_queue.cc:260] T 17168a1422cc4f7691a852382f021e82 P 56c48e5e70aa4d5a8df52fb119bbc087 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "752e3a6dba8942d4b3608e79bd812dd4" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 38803 } } peers { permanent_uuid: "d50490450e084a928b6a6e2fd376c87e" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 41811 } } peers { permanent_uuid: "56c48e5e70aa4d5a8df52fb119bbc087" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 41595 } }
I20260929 10:33:05.986333 16718 heartbeater.cc:503] Master 127.15.82.126:41933 was elected leader, sending a full tablet report...
I20260929 10:33:05.987573 16736 ts_tablet_manager.cc:1428] T 17168a1422cc4f7691a852382f021e82 P 56c48e5e70aa4d5a8df52fb119bbc087: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
W20260929 10:33:06.149374 16719 tablet.cc:2369] T 17168a1422cc4f7691a852382f021e82 P 56c48e5e70aa4d5a8df52fb119bbc087: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 10:33:06.178737 16585 tablet.cc:2369] T 17168a1422cc4f7691a852382f021e82 P d50490450e084a928b6a6e2fd376c87e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 10:33:06.213817 16451 tablet.cc:2369] T 17168a1422cc4f7691a852382f021e82 P 752e3a6dba8942d4b3608e79bd812dd4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 10:33:06.344826 16741 raft_consensus.cc:482] T 17168a1422cc4f7691a852382f021e82 P 752e3a6dba8942d4b3608e79bd812dd4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:33:06.345049 16741 raft_consensus.cc:504] T 17168a1422cc4f7691a852382f021e82 P 752e3a6dba8942d4b3608e79bd812dd4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "752e3a6dba8942d4b3608e79bd812dd4" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 38803 } } peers { permanent_uuid: "d50490450e084a928b6a6e2fd376c87e" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 41811 } } peers { permanent_uuid: "56c48e5e70aa4d5a8df52fb119bbc087" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 41595 } }
I20260929 10:33:06.346252 16741 leader_election.cc:288] T 17168a1422cc4f7691a852382f021e82 P 752e3a6dba8942d4b3608e79bd812dd4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d50490450e084a928b6a6e2fd376c87e (127.15.82.66:41811), 56c48e5e70aa4d5a8df52fb119bbc087 (127.15.82.67:41595)
I20260929 10:33:06.350425 16732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:06.346688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.65:35891 (local address 127.15.82.66:41811)
0929 10:33:06.346811 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:06.346815 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:06.347051 (+ 236us) server_negotiation.cc:407] Connection header received
0929 10:33:06.347155 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:06.347158 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:06.347216 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:06.347300 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:33:06.347650 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:06.348192 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:06.349433 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:06.349810 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:06.349943 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:06.349993 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:06.350189 (+ 196us) server_negotiation.cc:299] Negotiation successful
0929 10:33:06.350259 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260929 10:33:06.350433 16744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:06.346566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.67:41595 (local address 127.15.82.65:60427)
0929 10:33:06.347057 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:06.347072 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:06.347177 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:06.347449 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:06.347451 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:06.347464 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:33:06.347619 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:06.347626 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:06.348359 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:06.348365 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:06.349243 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:06.349251 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:06.349928 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:06.349931 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:06.350040 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:06.350083 (+ 43us) client_negotiation.cc:769] Sending connection context
0929 10:33:06.350172 (+ 89us) client_negotiation.cc:240] Negotiation successful
0929 10:33:06.350258 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":404,"mutex_wait_us":85,"thread_start_us":107,"threads_started":1}
I20260929 10:33:06.350421 16734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:06.346646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.65:60427 (local address 127.15.82.67:41595)
0929 10:33:06.346790 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:06.346794 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:06.347185 (+ 391us) server_negotiation.cc:407] Connection header received
0929 10:33:06.347256 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:06.347258 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:06.347338 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:06.347416 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:33:06.347738 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:06.348218 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:06.349370 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:06.349792 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:06.349913 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:06.349970 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:06.350194 (+ 224us) server_negotiation.cc:299] Negotiation successful
0929 10:33:06.350269 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260929 10:33:06.350467 16745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:06.346565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.66:41811 (local address 127.15.82.65:35891)
0929 10:33:06.346951 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:06.346964 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:06.347066 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:06.347330 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:06.347333 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:06.347349 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:33:06.347539 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:06.347546 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:06.348319 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:06.348322 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:06.349287 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:06.349322 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:06.349955 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:06.349959 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:06.350070 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:06.350088 (+ 18us) client_negotiation.cc:769] Sending connection context
0929 10:33:06.350174 (+ 86us) client_negotiation.cc:240] Negotiation successful
0929 10:33:06.350276 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"mutex_wait_us":94,"thread_start_us":116,"threads_started":1}
I20260929 10:33:06.351171 16538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17168a1422cc4f7691a852382f021e82" candidate_uuid: "752e3a6dba8942d4b3608e79bd812dd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d50490450e084a928b6a6e2fd376c87e" is_pre_election: true
I20260929 10:33:06.351176 16672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17168a1422cc4f7691a852382f021e82" candidate_uuid: "752e3a6dba8942d4b3608e79bd812dd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56c48e5e70aa4d5a8df52fb119bbc087" is_pre_election: true
I20260929 10:33:06.351500 16538 raft_consensus.cc:2454] T 17168a1422cc4f7691a852382f021e82 P d50490450e084a928b6a6e2fd376c87e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 752e3a6dba8942d4b3608e79bd812dd4 in term 0.
I20260929 10:33:06.351500 16672 raft_consensus.cc:2454] T 17168a1422cc4f7691a852382f021e82 P 56c48e5e70aa4d5a8df52fb119bbc087 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 752e3a6dba8942d4b3608e79bd812dd4 in term 0.
I20260929 10:33:06.352047 16340 leader_election.cc:302] T 17168a1422cc4f7691a852382f021e82 P 752e3a6dba8942d4b3608e79bd812dd4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 56c48e5e70aa4d5a8df52fb119bbc087, 752e3a6dba8942d4b3608e79bd812dd4, d50490450e084a928b6a6e2fd376c87e; no voters:
I20260929 10:33:06.352355 16741 raft_consensus.cc:2781] T 17168a1422cc4f7691a852382f021e82 P 752e3a6dba8942d4b3608e79bd812dd4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:33:06.352433 16741 raft_consensus.cc:482] T 17168a1422cc4f7691a852382f021e82 P 752e3a6dba8942d4b3608e79bd812dd4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:33:06.352464 16741 raft_consensus.cc:3037] T 17168a1422cc4f7691a852382f021e82 P 752e3a6dba8942d4b3608e79bd812dd4 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:33:06.353559 16741 raft_consensus.cc:504] T 17168a1422cc4f7691a852382f021e82 P 752e3a6dba8942d4b3608e79bd812dd4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "752e3a6dba8942d4b3608e79bd812dd4" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 38803 } } peers { permanent_uuid: "d50490450e084a928b6a6e2fd376c87e" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 41811 } } peers { permanent_uuid: "56c48e5e70aa4d5a8df52fb119bbc087" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 41595 } }
I20260929 10:33:06.353922 16741 leader_election.cc:288] T 17168a1422cc4f7691a852382f021e82 P 752e3a6dba8942d4b3608e79bd812dd4 [CANDIDATE]: Term 1 election: Requested vote from peers d50490450e084a928b6a6e2fd376c87e (127.15.82.66:41811), 56c48e5e70aa4d5a8df52fb119bbc087 (127.15.82.67:41595)
I20260929 10:33:06.354425 16672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17168a1422cc4f7691a852382f021e82" candidate_uuid: "752e3a6dba8942d4b3608e79bd812dd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56c48e5e70aa4d5a8df52fb119bbc087"
I20260929 10:33:06.354487 16538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17168a1422cc4f7691a852382f021e82" candidate_uuid: "752e3a6dba8942d4b3608e79bd812dd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d50490450e084a928b6a6e2fd376c87e"
I20260929 10:33:06.354578 16672 raft_consensus.cc:3037] T 17168a1422cc4f7691a852382f021e82 P 56c48e5e70aa4d5a8df52fb119bbc087 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:33:06.354604 16538 raft_consensus.cc:3037] T 17168a1422cc4f7691a852382f021e82 P d50490450e084a928b6a6e2fd376c87e [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:33:06.355969 16672 raft_consensus.cc:2454] T 17168a1422cc4f7691a852382f021e82 P 56c48e5e70aa4d5a8df52fb119bbc087 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 752e3a6dba8942d4b3608e79bd812dd4 in term 1.
I20260929 10:33:06.355969 16538 raft_consensus.cc:2454] T 17168a1422cc4f7691a852382f021e82 P d50490450e084a928b6a6e2fd376c87e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 752e3a6dba8942d4b3608e79bd812dd4 in term 1.
I20260929 10:33:06.356294 16340 leader_election.cc:302] T 17168a1422cc4f7691a852382f021e82 P 752e3a6dba8942d4b3608e79bd812dd4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56c48e5e70aa4d5a8df52fb119bbc087, 752e3a6dba8942d4b3608e79bd812dd4; no voters:
I20260929 10:33:06.356542 16741 raft_consensus.cc:2781] T 17168a1422cc4f7691a852382f021e82 P 752e3a6dba8942d4b3608e79bd812dd4 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:33:06.356765 16741 raft_consensus.cc:686] T 17168a1422cc4f7691a852382f021e82 P 752e3a6dba8942d4b3608e79bd812dd4 [term 1 LEADER]: Becoming Leader. State: Replica: 752e3a6dba8942d4b3608e79bd812dd4, State: Running, Role: LEADER
I20260929 10:33:06.357167 16741 consensus_queue.cc:237] T 17168a1422cc4f7691a852382f021e82 P 752e3a6dba8942d4b3608e79bd812dd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "752e3a6dba8942d4b3608e79bd812dd4" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 38803 } } peers { permanent_uuid: "d50490450e084a928b6a6e2fd376c87e" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 41811 } } peers { permanent_uuid: "56c48e5e70aa4d5a8df52fb119bbc087" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 41595 } }
I20260929 10:33:06.360841 16259 catalog_manager.cc:5495] T 17168a1422cc4f7691a852382f021e82 P 752e3a6dba8942d4b3608e79bd812dd4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 752e3a6dba8942d4b3608e79bd812dd4 (127.15.82.65). New cstate: current_term: 1 leader_uuid: "752e3a6dba8942d4b3608e79bd812dd4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "752e3a6dba8942d4b3608e79bd812dd4" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 38803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d50490450e084a928b6a6e2fd376c87e" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 41811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56c48e5e70aa4d5a8df52fb119bbc087" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 41595 } health_report { overall_health: UNKNOWN } } }
Sep 29 10:33:06 dist-test-slave-vg89 krb5kdc[16214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790677984, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.82.65@KRBTEST.COM
I20260929 10:33:06.413662 16731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:06.404486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42678 (local address 127.15.82.65:38803)
0929 10:33:06.404670 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:06.404675 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:06.404813 (+ 138us) server_negotiation.cc:407] Connection header received
0929 10:33:06.404985 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:06.404988 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:06.405049 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:06.405137 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:06.405846 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:06.406361 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:06.407269 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:06.407503 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:06.407615 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:06.410159 (+ 2544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:06.410175 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:06.410202 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:06.410227 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:06.411838 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:06.412290 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:06.412302 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:06.412303 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:06.412386 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:06.412672 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:06.412676 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:06.412678 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:06.412944 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:06.413178 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:06.413334 (+ 156us) server_negotiation.cc:299] Negotiation successful
0929 10:33:06.413489 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
Sep 29 10:33:06 dist-test-slave-vg89 krb5kdc[16214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790677986, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 29 10:33:06 dist-test-slave-vg89 krb5kdc[16214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790677986, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.82.65@KRBTEST.COM
I20260929 10:33:06.440670 16731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:06.432557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42682 (local address 127.15.82.65:38803)
0929 10:33:06.432748 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:06.432751 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:06.432817 (+ 66us) server_negotiation.cc:407] Connection header received
0929 10:33:06.432931 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:06.432933 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:06.432975 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:06.433062 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:06.433695 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:06.434219 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:06.435065 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:06.435290 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:06.435367 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:06.437838 (+ 2471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:06.437841 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:06.437843 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:06.437865 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:06.439181 (+ 1316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:06.439542 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:06.439546 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:06.439547 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:06.439630 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:06.439850 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:06.439852 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:06.439853 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:06.440009 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:06.440144 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:06.440356 (+ 212us) server_negotiation.cc:299] Negotiation successful
0929 10:33:06.440449 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
Sep 29 10:33:06 dist-test-slave-vg89 krb5kdc[16214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790677986, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 29 10:33:06 dist-test-slave-vg89 krb5kdc[16214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790677986, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.82.65@KRBTEST.COM
I20260929 10:33:06.467139 16731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:06.458318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42694 (local address 127.15.82.65:38803)
0929 10:33:06.458542 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:06.458546 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:06.458659 (+ 113us) server_negotiation.cc:407] Connection header received
0929 10:33:06.458825 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:06.458833 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:06.458874 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:06.458963 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:06.459763 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:06.460280 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:06.461170 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:06.461455 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:06.461570 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:06.463993 (+ 2423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:06.463997 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:06.464000 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:06.464033 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:06.465459 (+ 1426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:06.465880 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:06.465883 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:06.465885 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:06.465963 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:06.466235 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:06.466240 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:06.466242 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:06.466467 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:06.466627 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:06.466868 (+ 241us) server_negotiation.cc:299] Negotiation successful
0929 10:33:06.466985 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260929 10:33:06.468551 16364 tablet_service.cc:2908] Rejecting scan request for tablet 17168a1422cc4f7691a852382f021e82: Uninitialized: clean time has not yet been initialized
I20260929 10:33:06.470465 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 16321
I20260929 10:33:06.476603 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 16455
I20260929 10:33:06.482179 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 16589
I20260929 10:33:06.487821 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 16227
2026-09-29T10:33:06Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2638 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:33:06 dist-test-slave-vg89 krb5kdc[16760](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:33:06 dist-test-slave-vg89 krb5kdc[16760](info): set up 2 sockets
Sep 29 10:33:06 dist-test-slave-vg89 krb5kdc[16760](info): commencing operation
krb5kdc: starting...
W20260929 10:33:07.526616 15689 mini_kdc.cc:121] Time spent starting KDC: real 1.016s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Sep 29 10:33:07 dist-test-slave-vg89 krb5kdc[16760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790677987, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:33:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:33:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.82.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab.
Entry for principal kudu/127.15.82.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab.
I20260929 10:33:07.657619 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:40081
--webserver_interface=127.15.82.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab
--principal=kudu/127.15.82.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:38337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.82.126:40081
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:33:07.772195 16773 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:33:07.772541 16773 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:33:07.772619 16773 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:33:07.776731 16773 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:33:07.776840 16773 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260929 10:33:07.776871 16773 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:33:07.776912 16773 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:33:07.776966 16773 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:33:07.776997 16773 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:33:07.782610 16773 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:38337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.82.126:40081
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab
--principal=kudu/127.15.82.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:40081
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.82.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16773
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:33:07.783809 16773 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:33:07.784893 16773 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:33:07.793643 16778 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:07.793690 16779 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:07.793644 16781 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:07.794059 16773 server_base.cc:1034] running on GCE node
I20260929 10:33:07.794710 16773 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:33:07.795842 16773 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:33:07.797050 16773 hybrid_clock.cc:648] HybridClock initialized: now 1790677987797038 us; error 32 us; skew 500 ppm
Sep 29 10:33:07 dist-test-slave-vg89 krb5kdc[16760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790677987, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:33:07.804352 16773 init.cc:377] Logged in from keytab as kudu/127.15.82.126@KRBTEST.COM (short username kudu)
I20260929 10:33:07.805298 16773 webserver.cc:460] Webserver started at http://127.15.82.126:45371/ using document root <none> and password file <none>
I20260929 10:33:07.805891 16773 fs_manager.cc:359] Metadata directory not provided
I20260929 10:33:07.805984 16773 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:33:07.806255 16773 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:33:07.808408 16773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/minicluster-data/master-0/data/instance:
uuid: "c8c08b4557844cbbac9edc7f712c305f"
format_stamp: "Formatted at 2026-09-29 10:33:07 on dist-test-slave-vg89"
server_key: "ab0bd3fe875962460d8edf3e7996468f"
server_key_iv: "dcc4149ac7c89dd12e7ecfe15d6be0ed"
server_key_version: "encryptionkey@0"
I20260929 10:33:07.809080 16773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/minicluster-data/master-0/wal/instance:
uuid: "c8c08b4557844cbbac9edc7f712c305f"
format_stamp: "Formatted at 2026-09-29 10:33:07 on dist-test-slave-vg89"
server_key: "ab0bd3fe875962460d8edf3e7996468f"
server_key_iv: "dcc4149ac7c89dd12e7ecfe15d6be0ed"
server_key_version: "encryptionkey@0"
I20260929 10:33:07.813238 16773 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260929 10:33:07.815778 16788 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:07.817049 16773 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260929 10:33:07.817193 16773 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/minicluster-data/master-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/minicluster-data/master-0/wal
uuid: "c8c08b4557844cbbac9edc7f712c305f"
format_stamp: "Formatted at 2026-09-29 10:33:07 on dist-test-slave-vg89"
server_key: "ab0bd3fe875962460d8edf3e7996468f"
server_key_iv: "dcc4149ac7c89dd12e7ecfe15d6be0ed"
server_key_version: "encryptionkey@0"
I20260929 10:33:07.817309 16773 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:33:07.827086 16773 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:33:07.829746 16773 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:33:07.829942 16773 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:33:07.837601 16773 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.126:40081
I20260929 10:33:07.837625 16850 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.126:40081 every 8 connection(s)
I20260929 10:33:07.838544 16773 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
I20260929 10:33:07.841392 16851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:33:07.844480 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 16773
I20260929 10:33:07.844580 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/minicluster-data/master-0/wal/instance
I20260929 10:33:07.844844 15689 external_mini_cluster.cc:1428] Setting key 8121f9d4ad73486c27a4f51453bc6ca5
I20260929 10:33:07.847527 16851 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8c08b4557844cbbac9edc7f712c305f: Bootstrap starting.
I20260929 10:33:07.850365 16851 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c8c08b4557844cbbac9edc7f712c305f: Neither blocks nor log segments found. Creating new log.
I20260929 10:33:07.851236 16851 log.cc:824] T 00000000000000000000000000000000 P c8c08b4557844cbbac9edc7f712c305f: Log is configured to *not* fsync() on all Append() calls
I20260929 10:33:07.853252 16851 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8c08b4557844cbbac9edc7f712c305f: No bootstrap required, opened a new log
Sep 29 10:33:07 dist-test-slave-vg89 krb5kdc[16760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790677987, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:33:07.857100 16851 raft_consensus.cc:348] T 00000000000000000000000000000000 P c8c08b4557844cbbac9edc7f712c305f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8c08b4557844cbbac9edc7f712c305f" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 40081 } }
I20260929 10:33:07.857364 16851 raft_consensus.cc:374] T 00000000000000000000000000000000 P c8c08b4557844cbbac9edc7f712c305f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:33:07.857457 16851 raft_consensus.cc:729] T 00000000000000000000000000000000 P c8c08b4557844cbbac9edc7f712c305f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8c08b4557844cbbac9edc7f712c305f, State: Initialized, Role: FOLLOWER
I20260929 10:33:07.857969 16851 consensus_queue.cc:260] T 00000000000000000000000000000000 P c8c08b4557844cbbac9edc7f712c305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8c08b4557844cbbac9edc7f712c305f" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 40081 } }
I20260929 10:33:07.858160 16851 raft_consensus.cc:388] T 00000000000000000000000000000000 P c8c08b4557844cbbac9edc7f712c305f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:33:07.858237 16851 raft_consensus.cc:482] T 00000000000000000000000000000000 P c8c08b4557844cbbac9edc7f712c305f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:33:07.858338 16851 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c8c08b4557844cbbac9edc7f712c305f [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:33:07.859469 16851 raft_consensus.cc:504] T 00000000000000000000000000000000 P c8c08b4557844cbbac9edc7f712c305f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8c08b4557844cbbac9edc7f712c305f" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 40081 } }
I20260929 10:33:07.859819 16851 leader_election.cc:302] T 00000000000000000000000000000000 P c8c08b4557844cbbac9edc7f712c305f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c8c08b4557844cbbac9edc7f712c305f; no voters:
I20260929 10:33:07.859894 16854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:07.846298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46386 (local address 127.15.82.126:40081)
0929 10:33:07.846771 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:07.846790 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:07.846832 (+ 42us) server_negotiation.cc:407] Connection header received
0929 10:33:07.847650 (+ 818us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:07.847673 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:07.848119 (+ 446us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:07.848478 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:07.849219 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:07.850780 (+ 1561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:07.851696 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:07.852177 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:07.852365 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:07.854877 (+ 2512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:07.854882 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:07.854894 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:07.854920 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:07.856659 (+ 1739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:07.857119 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:07.857124 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:07.857129 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:07.857240 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:07.857543 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:07.857547 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:07.857548 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:07.858049 (+ 501us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:07.858267 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:07.858600 (+ 333us) server_negotiation.cc:299] Negotiation successful
0929 10:33:07.858901 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":129,"threads_started":1}
I20260929 10:33:07.860185 16851 leader_election.cc:288] T 00000000000000000000000000000000 P c8c08b4557844cbbac9edc7f712c305f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:33:07.860286 16856 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c8c08b4557844cbbac9edc7f712c305f [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:33:07.860528 16856 raft_consensus.cc:686] T 00000000000000000000000000000000 P c8c08b4557844cbbac9edc7f712c305f [term 1 LEADER]: Becoming Leader. State: Replica: c8c08b4557844cbbac9edc7f712c305f, State: Running, Role: LEADER
I20260929 10:33:07.860877 16856 consensus_queue.cc:237] T 00000000000000000000000000000000 P c8c08b4557844cbbac9edc7f712c305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8c08b4557844cbbac9edc7f712c305f" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 40081 } }
I20260929 10:33:07.861415 16851 sys_catalog.cc:564] T 00000000000000000000000000000000 P c8c08b4557844cbbac9edc7f712c305f [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:33:07.862747 16857 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8c08b4557844cbbac9edc7f712c305f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c8c08b4557844cbbac9edc7f712c305f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8c08b4557844cbbac9edc7f712c305f" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 40081 } } }
I20260929 10:33:07.862870 16857 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8c08b4557844cbbac9edc7f712c305f [sys.catalog]: This master's current role is: LEADER
I20260929 10:33:07.863261 16866 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:33:07.863260 16858 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8c08b4557844cbbac9edc7f712c305f [sys.catalog]: SysCatalogTable state changed. Reason: New leader c8c08b4557844cbbac9edc7f712c305f. Latest consensus state: current_term: 1 leader_uuid: "c8c08b4557844cbbac9edc7f712c305f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8c08b4557844cbbac9edc7f712c305f" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 40081 } } }
I20260929 10:33:07.863389 16858 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8c08b4557844cbbac9edc7f712c305f [sys.catalog]: This master's current role is: LEADER
I20260929 10:33:07.866631 16866 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:33:07.872318 16866 catalog_manager.cc:1347] Generated new cluster ID: 06105c5ab3b9459188cafdab479e392e
I20260929 10:33:07.872380 16866 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:33:07.881546 16866 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:33:07.882544 16866 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:33:07.888810 16866 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c8c08b4557844cbbac9edc7f712c305f: Generated new TSK 0
I20260929 10:33:07.889532 16866 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.82.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab.
Entry for principal kudu/127.15.82.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab.
I20260929 10:33:07.921492 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.65:0
--local_ip_for_outbound_sockets=127.15.82.65
--webserver_interface=127.15.82.65
--webserver_port=0
--tserver_master_addrs=127.15.82.126:40081
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab
--principal=kudu/127.15.82.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:38337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:33:08.036656 16877 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:33:08.036942 16877 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:33:08.036986 16877 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:33:08.041057 16877 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:33:08.041114 16877 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:33:08.041152 16877 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 10:33:08.041255 16877 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.65
I20260929 10:33:08.046418 16877 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:38337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab
--principal=kudu/127.15.82.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.82.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.82.126:40081
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16877
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.65
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:33:08.047456 16877 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:33:08.048524 16877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:33:08.058115 16885 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:08.058279 16877 server_base.cc:1034] running on GCE node
W20260929 10:33:08.058131 16882 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:08.058182 16883 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:08.058816 16877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:33:08.059393 16877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:33:08.060568 16877 hybrid_clock.cc:648] HybridClock initialized: now 1790677988060552 us; error 28 us; skew 500 ppm
Sep 29 10:33:08 dist-test-slave-vg89 krb5kdc[16760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790677988, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:33:08.063510 16877 init.cc:377] Logged in from keytab as kudu/127.15.82.65@KRBTEST.COM (short username kudu)
I20260929 10:33:08.064335 16877 webserver.cc:460] Webserver started at http://127.15.82.65:44727/ using document root <none> and password file <none>
I20260929 10:33:08.064899 16877 fs_manager.cc:359] Metadata directory not provided
I20260929 10:33:08.064989 16877 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:33:08.065214 16877 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:33:08.067298 16877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/minicluster-data/ts-0/data/instance:
uuid: "c8405f63549b4e06a48843842f6e1d3c"
format_stamp: "Formatted at 2026-09-29 10:33:08 on dist-test-slave-vg89"
server_key: "d1a5328fcd79b2027e596b5716ff8223"
server_key_iv: "f399c2a8128883f1d195ecaf5c543114"
server_key_version: "encryptionkey@0"
I20260929 10:33:08.067898 16877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/minicluster-data/ts-0/wal/instance:
uuid: "c8405f63549b4e06a48843842f6e1d3c"
format_stamp: "Formatted at 2026-09-29 10:33:08 on dist-test-slave-vg89"
server_key: "d1a5328fcd79b2027e596b5716ff8223"
server_key_iv: "f399c2a8128883f1d195ecaf5c543114"
server_key_version: "encryptionkey@0"
I20260929 10:33:08.071587 16877 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260929 10:33:08.074046 16892 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:08.075332 16877 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 10:33:08.075469 16877 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/minicluster-data/ts-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/minicluster-data/ts-0/wal
uuid: "c8405f63549b4e06a48843842f6e1d3c"
format_stamp: "Formatted at 2026-09-29 10:33:08 on dist-test-slave-vg89"
server_key: "d1a5328fcd79b2027e596b5716ff8223"
server_key_iv: "f399c2a8128883f1d195ecaf5c543114"
server_key_version: "encryptionkey@0"
I20260929 10:33:08.075603 16877 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:33:08.092012 16877 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:33:08.095021 16877 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:33:08.095229 16877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:33:08.096692 16877 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:33:08.096769 16877 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:08.096839 16877 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:33:08.096890 16877 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 10:33:08 dist-test-slave-vg89 krb5kdc[16760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790677988, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.65@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:33:08.107975 16877 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.65:43109
I20260929 10:33:08.108040 17007 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.65:43109 every 8 connection(s)
I20260929 10:33:08.108951 16877 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
I20260929 10:33:08.111693 16854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:08.097583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.65:41583 (local address 127.15.82.126:40081)
0929 10:33:08.097752 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:08.097757 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:08.098717 (+ 960us) server_negotiation.cc:407] Connection header received
0929 10:33:08.099841 (+ 1124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:08.099845 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:08.099905 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:08.100036 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:08.101699 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:08.102608 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:08.104538 (+ 1930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:08.104865 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:08.104949 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:08.108186 (+ 3237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:08.108190 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:08.108193 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:08.108228 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:08.109881 (+ 1653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:08.110316 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:08.110318 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:08.110320 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:08.110391 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:08.110707 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:08.110709 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:08.110710 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:08.110888 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:08.110965 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:08.111458 (+ 493us) server_negotiation.cc:299] Negotiation successful
0929 10:33:08.111571 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260929 10:33:08.112490 16901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:08.098025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:40081 (local address 127.15.82.65:41583)
0929 10:33:08.098557 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:08.098606 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:08.099574 (+ 968us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:08.100273 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:08.100283 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:08.100787 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:08.101515 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:08.101539 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:08.102755 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:08.102760 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:08.104366 (+ 1606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:08.104379 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:08.105032 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:08.105036 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:08.105258 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:08.106063 (+ 805us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:08.106088 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:08.107953 (+ 1865us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:08.109975 (+ 2022us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:08.109979 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:08.109990 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:08.110233 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:08.110500 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:08.110502 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:08.110503 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:08.110627 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:08.110982 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:08.110986 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:08.111205 (+ 219us) client_negotiation.cc:769] Sending connection context
0929 10:33:08.111425 (+ 220us) client_negotiation.cc:240] Negotiation successful
0929 10:33:08.111674 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":144,"threads_started":1}
I20260929 10:33:08.113874 17008 heartbeater.cc:347] Connected to a master server at 127.15.82.126:40081
I20260929 10:33:08.114034 17008 heartbeater.cc:464] Registering TS with master...
I20260929 10:33:08.114593 17008 heartbeater.cc:511] Master 127.15.82.126:40081 requested a full tablet report, sending...
I20260929 10:33:08.116210 16804 ts_manager.cc:194] Registered new tserver with Master: c8405f63549b4e06a48843842f6e1d3c (127.15.82.65:43109)
I20260929 10:33:08.117537 16804 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.82.65@KRBTEST.COM'} at 127.15.82.65:41583
I20260929 10:33:08.118983 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 16877
I20260929 10:33:08.119100 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/minicluster-data/ts-0/wal/instance
I20260929 10:33:08.119413 15689 external_mini_cluster.cc:1428] Setting key fb8f18a5e75398285473417d3cd5a809
I20260929 10:33:08.123512 16854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:08.116011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.65:51847 (local address 127.15.82.126:40081)
0929 10:33:08.116143 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:08.116147 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:08.116347 (+ 200us) server_negotiation.cc:407] Connection header received
0929 10:33:08.116471 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:08.116474 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:08.116511 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:08.116584 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:08.117057 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:08.117600 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:08.118683 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:08.118890 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:08.119001 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:08.120239 (+ 1238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:08.120243 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:08.120246 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:08.120269 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:08.121769 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:08.122187 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:08.122191 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:08.122193 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:08.122269 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:08.122607 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:08.122612 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:08.122616 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:08.122831 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:08.122916 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:08.123256 (+ 340us) server_negotiation.cc:299] Negotiation successful
0929 10:33:08.123403 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260929 10:33:08.123620 17014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:08.115934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:40081 (local address 127.15.82.65:51847)
0929 10:33:08.116255 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:08.116271 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:08.116376 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:08.116628 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:08.116631 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:08.116828 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:08.116948 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:08.116955 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:08.117750 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:08.117753 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:08.118556 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:08.118566 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:08.119003 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:08.119006 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:08.119142 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:08.119651 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:08.119662 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:08.120110 (+ 448us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:08.121874 (+ 1764us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:08.121878 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:08.121880 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:08.122087 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:08.122361 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:08.122364 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:08.122366 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:08.122468 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:08.122929 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:08.122933 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:08.123048 (+ 115us) client_negotiation.cc:769] Sending connection context
0929 10:33:08.123206 (+ 158us) client_negotiation.cc:240] Negotiation successful
0929 10:33:08.123370 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"mutex_wait_us":6,"thread_start_us":135,"threads_started":1}
WARNING: no policy specified for kudu/127.15.82.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab.
Entry for principal kudu/127.15.82.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab.
I20260929 10:33:08.146770 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.66:0
--local_ip_for_outbound_sockets=127.15.82.66
--webserver_interface=127.15.82.66
--webserver_port=0
--tserver_master_addrs=127.15.82.126:40081
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab
--principal=kudu/127.15.82.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:38337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:33:08.258876 17017 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:33:08.259140 17017 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:33:08.259186 17017 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:33:08.263501 17017 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:33:08.263567 17017 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:33:08.263607 17017 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 10:33:08.263641 17017 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.66
I20260929 10:33:08.269065 17017 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:38337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab
--principal=kudu/127.15.82.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.82.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.82.126:40081
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17017
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.66
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:33:08.270107 17017 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:33:08.271190 17017 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:33:08.281005 17025 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:08.280995 17022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:08.280995 17023 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:08.281343 17017 server_base.cc:1034] running on GCE node
I20260929 10:33:08.281738 17017 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:33:08.282555 17017 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:33:08.283720 17017 hybrid_clock.cc:648] HybridClock initialized: now 1790677988283695 us; error 38 us; skew 500 ppm
Sep 29 10:33:08 dist-test-slave-vg89 krb5kdc[16760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790677988, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:33:08.286633 17017 init.cc:377] Logged in from keytab as kudu/127.15.82.66@KRBTEST.COM (short username kudu)
I20260929 10:33:08.287510 17017 webserver.cc:460] Webserver started at http://127.15.82.66:45245/ using document root <none> and password file <none>
I20260929 10:33:08.288094 17017 fs_manager.cc:359] Metadata directory not provided
I20260929 10:33:08.288153 17017 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:33:08.288365 17017 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:33:08.290488 17017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/minicluster-data/ts-1/data/instance:
uuid: "86dae52b03fd4ebab95bbdb5d4a45d8a"
format_stamp: "Formatted at 2026-09-29 10:33:08 on dist-test-slave-vg89"
server_key: "5f4c487025a85714366b97581964a443"
server_key_iv: "9e934f98a05648521cc5a5f06df89c28"
server_key_version: "encryptionkey@0"
I20260929 10:33:08.291067 17017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/minicluster-data/ts-1/wal/instance:
uuid: "86dae52b03fd4ebab95bbdb5d4a45d8a"
format_stamp: "Formatted at 2026-09-29 10:33:08 on dist-test-slave-vg89"
server_key: "5f4c487025a85714366b97581964a443"
server_key_iv: "9e934f98a05648521cc5a5f06df89c28"
server_key_version: "encryptionkey@0"
I20260929 10:33:08.294764 17017 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20260929 10:33:08.297158 17032 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:08.298448 17017 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 10:33:08.298593 17017 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/minicluster-data/ts-1/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/minicluster-data/ts-1/wal
uuid: "86dae52b03fd4ebab95bbdb5d4a45d8a"
format_stamp: "Formatted at 2026-09-29 10:33:08 on dist-test-slave-vg89"
server_key: "5f4c487025a85714366b97581964a443"
server_key_iv: "9e934f98a05648521cc5a5f06df89c28"
server_key_version: "encryptionkey@0"
I20260929 10:33:08.298699 17017 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:33:08.312770 17017 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:33:08.316332 17017 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:33:08.316555 17017 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:33:08.318080 17017 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:33:08.318157 17017 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:08.318219 17017 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:33:08.318249 17017 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 10:33:08 dist-test-slave-vg89 krb5kdc[16760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790677988, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.66@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:33:08.328792 17017 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.66:34809
I20260929 10:33:08.328857 17147 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.66:34809 every 8 connection(s)
I20260929 10:33:08.329914 17017 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
I20260929 10:33:08.332899 16854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:08.318933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:41245 (local address 127.15.82.126:40081)
0929 10:33:08.319109 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:08.319116 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:08.319952 (+ 836us) server_negotiation.cc:407] Connection header received
0929 10:33:08.321184 (+ 1232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:08.321189 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:08.321258 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:08.321366 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:08.322975 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:08.323899 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:08.325739 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:08.326087 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:08.326228 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:08.329258 (+ 3030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:08.329269 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:08.329273 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:08.329307 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:08.330950 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:08.331476 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:08.331479 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:08.331481 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:08.331567 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:08.331937 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:08.331940 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:08.331941 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:08.332136 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:08.332217 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:08.332679 (+ 462us) server_negotiation.cc:299] Negotiation successful
0929 10:33:08.332781 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260929 10:33:08.333177 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 17017
I20260929 10:33:08.333304 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/minicluster-data/ts-1/wal/instance
I20260929 10:33:08.333600 15689 external_mini_cluster.cc:1428] Setting key 7566625a0f827d3e1c41bd72334e8e69
I20260929 10:33:08.333853 17041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:08.319329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:40081 (local address 127.15.82.66:41245)
0929 10:33:08.319779 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:08.319830 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:08.320878 (+ 1048us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:08.321574 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:08.321583 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:08.322032 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:08.322795 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:08.322817 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:08.324038 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:08.324042 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:08.325596 (+ 1554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:08.325608 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:08.326243 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:08.326246 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:08.326386 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:08.327155 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:08.327183 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:08.329095 (+ 1912us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:08.331079 (+ 1984us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:08.331085 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:08.331100 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:08.331363 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:08.331673 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:08.331676 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:08.331678 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:08.331834 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:08.332238 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:08.332243 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:08.332471 (+ 228us) client_negotiation.cc:769] Sending connection context
0929 10:33:08.332628 (+ 157us) client_negotiation.cc:240] Negotiation successful
0929 10:33:08.332877 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":131,"threads_started":1}
I20260929 10:33:08.335201 17148 heartbeater.cc:347] Connected to a master server at 127.15.82.126:40081
I20260929 10:33:08.335352 17148 heartbeater.cc:464] Registering TS with master...
I20260929 10:33:08.335948 17148 heartbeater.cc:511] Master 127.15.82.126:40081 requested a full tablet report, sending...
I20260929 10:33:08.337080 16804 ts_manager.cc:194] Registered new tserver with Master: 86dae52b03fd4ebab95bbdb5d4a45d8a (127.15.82.66:34809)
I20260929 10:33:08.337858 16804 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.82.66@KRBTEST.COM'} at 127.15.82.66:41245
WARNING: no policy specified for kudu/127.15.82.67@KRBTEST.COM; defaulting to no policy
I20260929 10:33:08.345314 16854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:08.337569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:51617 (local address 127.15.82.126:40081)
0929 10:33:08.337660 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:08.337664 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:08.337677 (+ 13us) server_negotiation.cc:407] Connection header received
0929 10:33:08.337770 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:08.337773 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:08.337815 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:08.337898 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:08.338451 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:08.339189 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:08.340585 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:08.340791 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:08.340871 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:08.342216 (+ 1345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:08.342219 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:08.342221 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:08.342242 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:08.343748 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:08.344232 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:08.344234 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:08.344235 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:08.344306 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:08.344598 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:08.344600 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:08.344601 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:08.344803 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:08.344879 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:08.345107 (+ 228us) server_negotiation.cc:299] Negotiation successful
0929 10:33:08.345196 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260929 10:33:08.345388 17155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:08.337236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:40081 (local address 127.15.82.66:51617)
0929 10:33:08.337555 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:08.337574 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:08.337667 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:08.337929 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:08.337940 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:08.338166 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:08.338302 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:08.338311 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:08.339340 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:08.339346 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:08.340452 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:08.340467 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:08.340910 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:08.340916 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:08.341058 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:08.341577 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:08.341597 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:08.342074 (+ 477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:08.343864 (+ 1790us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:08.343871 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:08.343875 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:08.344140 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:08.344400 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:08.344406 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:08.344410 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:08.344516 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:08.344897 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:08.344900 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:08.344989 (+ 89us) client_negotiation.cc:769] Sending connection context
0929 10:33:08.345071 (+ 82us) client_negotiation.cc:240] Negotiation successful
0929 10:33:08.345181 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"mutex_wait_us":4,"thread_start_us":111,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab.
Entry for principal kudu/127.15.82.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab.
I20260929 10:33:08.363304 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.67:0
--local_ip_for_outbound_sockets=127.15.82.67
--webserver_interface=127.15.82.67
--webserver_port=0
--tserver_master_addrs=127.15.82.126:40081
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab
--principal=kudu/127.15.82.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:38337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:33:08.475266 17157 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:33:08.475581 17157 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:33:08.475665 17157 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:33:08.479735 17157 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:33:08.479874 17157 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:33:08.479930 17157 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 10:33:08.479964 17157 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.67
I20260929 10:33:08.485062 17157 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:38337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab
--principal=kudu/127.15.82.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.82.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.82.126:40081
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17157
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.67
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:33:08.486032 17157 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:33:08.487072 17157 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:33:08.496973 17162 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:08.497038 17163 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:08.497038 17165 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:08.497300 17157 server_base.cc:1034] running on GCE node
I20260929 10:33:08.497723 17157 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:33:08.498456 17157 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:33:08.499642 17157 hybrid_clock.cc:648] HybridClock initialized: now 1790677988499632 us; error 33 us; skew 500 ppm
Sep 29 10:33:08 dist-test-slave-vg89 krb5kdc[16760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790677988, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:33:08.502559 17157 init.cc:377] Logged in from keytab as kudu/127.15.82.67@KRBTEST.COM (short username kudu)
I20260929 10:33:08.503525 17157 webserver.cc:460] Webserver started at http://127.15.82.67:44059/ using document root <none> and password file <none>
I20260929 10:33:08.504102 17157 fs_manager.cc:359] Metadata directory not provided
I20260929 10:33:08.504194 17157 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:33:08.504423 17157 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:33:08.506507 17157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/minicluster-data/ts-2/data/instance:
uuid: "67856bddf7404c0c99446a985257bc7e"
format_stamp: "Formatted at 2026-09-29 10:33:08 on dist-test-slave-vg89"
server_key: "b0098e331c63d1cd039889b254238275"
server_key_iv: "525acfb87daf2fbc13d9c2899f2eb019"
server_key_version: "encryptionkey@0"
I20260929 10:33:08.507073 17157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/minicluster-data/ts-2/wal/instance:
uuid: "67856bddf7404c0c99446a985257bc7e"
format_stamp: "Formatted at 2026-09-29 10:33:08 on dist-test-slave-vg89"
server_key: "b0098e331c63d1cd039889b254238275"
server_key_iv: "525acfb87daf2fbc13d9c2899f2eb019"
server_key_version: "encryptionkey@0"
I20260929 10:33:08.510886 17157 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260929 10:33:08.513618 17172 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:08.514954 17157 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 10:33:08.515089 17157 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/minicluster-data/ts-2/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/minicluster-data/ts-2/wal
uuid: "67856bddf7404c0c99446a985257bc7e"
format_stamp: "Formatted at 2026-09-29 10:33:08 on dist-test-slave-vg89"
server_key: "b0098e331c63d1cd039889b254238275"
server_key_iv: "525acfb87daf2fbc13d9c2899f2eb019"
server_key_version: "encryptionkey@0"
I20260929 10:33:08.515202 17157 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:33:08.528846 17157 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:33:08.531916 17157 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:33:08.532119 17157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:33:08.533555 17157 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:33:08.533641 17157 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:08.533717 17157 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:33:08.533771 17157 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 10:33:08 dist-test-slave-vg89 krb5kdc[16760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790677988, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.67@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:33:08.545080 17157 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.67:36029
I20260929 10:33:08.545357 17287 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.67:36029 every 8 connection(s)
I20260929 10:33:08.546300 17157 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
I20260929 10:33:08.548353 16854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:08.534485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.67:36857 (local address 127.15.82.126:40081)
0929 10:33:08.534616 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:08.534619 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:08.535760 (+ 1141us) server_negotiation.cc:407] Connection header received
0929 10:33:08.536906 (+ 1146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:08.536912 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:08.536971 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:08.537070 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:08.538832 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:08.539502 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:08.540823 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:08.541113 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:08.541231 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:08.544304 (+ 3073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:08.544306 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:08.544308 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:08.544335 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:08.546084 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:08.546687 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:08.546691 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:08.546693 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:08.546797 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:08.547201 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:08.547204 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:08.547206 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:08.547415 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:08.547528 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:08.548039 (+ 511us) server_negotiation.cc:299] Negotiation successful
0929 10:33:08.548167 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260929 10:33:08.549113 17181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:08.535057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:40081 (local address 127.15.82.67:36857)
0929 10:33:08.535612 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:08.535657 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:08.536645 (+ 988us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:08.537518 (+ 873us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:08.537527 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:08.537966 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:08.538685 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:08.538701 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:08.539630 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:08.539633 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:08.540684 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:08.540694 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:08.541283 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:08.541286 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:08.541514 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:08.542162 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:08.542178 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:08.544112 (+ 1934us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:08.546198 (+ 2086us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:08.546203 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:08.546215 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:08.546562 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:08.546921 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:08.546925 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:08.546927 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:08.547109 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:08.547525 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:08.547533 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:08.547782 (+ 249us) client_negotiation.cc:769] Sending connection context
0929 10:33:08.547984 (+ 202us) client_negotiation.cc:240] Negotiation successful
0929 10:33:08.548304 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":163,"threads_started":1}
I20260929 10:33:08.549906 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 17157
I20260929 10:33:08.550045 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/minicluster-data/ts-2/wal/instance
I20260929 10:33:08.550312 15689 external_mini_cluster.cc:1428] Setting key 9a23a4193649fbe729b2a3987e09a85f
I20260929 10:33:08.550571 17288 heartbeater.cc:347] Connected to a master server at 127.15.82.126:40081
I20260929 10:33:08.550721 17288 heartbeater.cc:464] Registering TS with master...
I20260929 10:33:08.551301 17288 heartbeater.cc:511] Master 127.15.82.126:40081 requested a full tablet report, sending...
I20260929 10:33:08.552450 16805 ts_manager.cc:194] Registered new tserver with Master: 67856bddf7404c0c99446a985257bc7e (127.15.82.67:36029)
I20260929 10:33:08.553241 16805 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.82.67@KRBTEST.COM'} at 127.15.82.67:36857
I20260929 10:33:08.559793 16854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:08.552805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.67:55379 (local address 127.15.82.126:40081)
0929 10:33:08.552933 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:08.552937 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:08.553109 (+ 172us) server_negotiation.cc:407] Connection header received
0929 10:33:08.553211 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:08.553213 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:08.553250 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:08.553312 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:08.553802 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:08.554586 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:08.555429 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:08.555817 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:08.555898 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:08.556889 (+ 991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:08.556892 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:08.556894 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:08.556915 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:08.558486 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:08.558863 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:08.558865 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:08.558867 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:08.558936 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:08.559158 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:08.559160 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:08.559161 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:08.559311 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:08.559385 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:08.559599 (+ 214us) server_negotiation.cc:299] Negotiation successful
0929 10:33:08.559682 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260929 10:33:08.559803 17294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:08.552744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:40081 (local address 127.15.82.67:55379)
0929 10:33:08.553022 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:08.553037 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:08.553127 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:08.553355 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:08.553358 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:08.553568 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:08.553695 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:08.553702 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:08.554734 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:08.554737 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:08.555293 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:08.555301 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:08.555925 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:08.555928 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:08.556002 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:08.556440 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:08.556452 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:08.556770 (+ 318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:08.558598 (+ 1828us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:08.558601 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:08.558602 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:08.558784 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:08.559011 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:08.559013 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:08.559014 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:08.559085 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:08.559413 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:08.559415 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:08.559487 (+ 72us) client_negotiation.cc:769] Sending connection context
0929 10:33:08.559563 (+ 76us) client_negotiation.cc:240] Negotiation successful
0929 10:33:08.559663 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":115,"threads_started":1}
I20260929 10:33:08.564661 15689 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 29 10:33:08 dist-test-slave-vg89 krb5kdc[16760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790677988, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 29 10:33:08 dist-test-slave-vg89 krb5kdc[16760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790677988, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:33:08.591379 16854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:08.583315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46398 (local address 127.15.82.126:40081)
0929 10:33:08.583462 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:08.583465 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:08.583478 (+ 13us) server_negotiation.cc:407] Connection header received
0929 10:33:08.583521 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:08.583523 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:08.583563 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:08.583660 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:08.584401 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:08.584932 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:08.585786 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:08.586016 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:08.586128 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:08.588487 (+ 2359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:08.588490 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:08.588493 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:08.588517 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:08.589711 (+ 1194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:08.590123 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:08.590126 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:08.590128 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:08.590210 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:08.590516 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:08.590522 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:08.590526 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:08.590703 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:08.590844 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:08.591088 (+ 244us) server_negotiation.cc:299] Negotiation successful
0929 10:33:08.591205 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 10:33:08.594187 16805 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46398:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 10:33:08.597182 16805 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:33:08.611469 17306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:08.605604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37520 (local address 127.15.82.66:34809)
0929 10:33:08.606596 (+ 992us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:08.606609 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:08.606634 (+ 25us) server_negotiation.cc:407] Connection header received
0929 10:33:08.606720 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:08.606724 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:08.606944 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:08.607127 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:33:08.607433 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:08.608374 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:08.609818 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:08.610722 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:08.610831 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:08.611007 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:08.611199 (+ 192us) server_negotiation.cc:299] Negotiation successful
0929 10:33:08.611296 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":890,"thread_start_us":114,"threads_started":1}
I20260929 10:33:08.611471 17305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:08.605461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.66:34809 (local address 127.0.0.1:37520)
0929 10:33:08.605899 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:08.605955 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:08.606079 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:08.607128 (+ 1049us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:08.607137 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:08.607163 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:33:08.607313 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:08.607319 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:08.608487 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:08.608491 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:08.609666 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:08.609677 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:08.610854 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:08.610858 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:08.610985 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:08.611029 (+ 44us) client_negotiation.cc:769] Sending connection context
0929 10:33:08.611151 (+ 122us) client_negotiation.cc:240] Negotiation successful
0929 10:33:08.611283 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"mutex_wait_us":3,"thread_start_us":102,"threads_started":1}
I20260929 10:33:08.611850 17308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:08.605462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.67:36029 (local address 127.0.0.1:56086)
0929 10:33:08.606299 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:08.606314 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:08.606867 (+ 553us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:08.607560 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:08.607564 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:08.607584 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:33:08.607676 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:08.607682 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:08.609156 (+ 1474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:08.609159 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:08.610368 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:08.610379 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:08.611460 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:08.611464 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:08.611559 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:08.611586 (+ 27us) client_negotiation.cc:769] Sending connection context
0929 10:33:08.611641 (+ 55us) client_negotiation.cc:240] Negotiation successful
0929 10:33:08.611709 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":774,"mutex_wait_us":567,"thread_start_us":98,"threads_started":1}
I20260929 10:33:08.612136 17309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:08.606081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.65:43109 (local address 127.0.0.1:45714)
0929 10:33:08.606776 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:08.606789 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:08.606871 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:08.607719 (+ 848us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:08.607722 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:08.607741 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:33:08.607821 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:08.607826 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:08.609183 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:08.609186 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:08.610382 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:08.610392 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:08.611711 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:08.611715 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:08.611807 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:08.611828 (+ 21us) client_negotiation.cc:769] Sending connection context
0929 10:33:08.611904 (+ 76us) client_negotiation.cc:240] Negotiation successful
0929 10:33:08.611992 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"mutex_wait_us":47,"thread_start_us":84,"threads_started":1}
I20260929 10:33:08.612229 17310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:08.606484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45714 (local address 127.15.82.65:43109)
0929 10:33:08.607129 (+ 645us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:08.607145 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:08.607168 (+ 23us) server_negotiation.cc:407] Connection header received
0929 10:33:08.607240 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:08.607245 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:08.607450 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:08.607630 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:33:08.607947 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:08.609051 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:08.610555 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:08.611342 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:08.611443 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:08.611643 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:08.611976 (+ 333us) server_negotiation.cc:299] Negotiation successful
0929 10:33:08.612086 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":511,"thread_start_us":90,"threads_started":1}
I20260929 10:33:08.612632 17307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:08.605714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56086 (local address 127.15.82.67:36029)
0929 10:33:08.606596 (+ 882us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:08.606609 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:08.606634 (+ 25us) server_negotiation.cc:407] Connection header received
0929 10:33:08.607218 (+ 584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:08.607223 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:08.607412 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:08.607562 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:33:08.607799 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:08.609028 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:08.610498 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:08.611327 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:08.612145 (+ 818us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:08.612311 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:08.612369 (+ 58us) server_negotiation.cc:299] Negotiation successful
0929 10:33:08.612497 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":777,"thread_start_us":115,"threads_started":1}
I20260929 10:33:08.615130 17222 tablet_service.cc:1467] Processing CreateTablet for tablet c058b842d7c845b0966d26e475fa8a50 (DEFAULT_TABLE table=test-table [id=874eea9c9921451ca790ae8eb3126247]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:33:08.615140 17082 tablet_service.cc:1467] Processing CreateTablet for tablet c058b842d7c845b0966d26e475fa8a50 (DEFAULT_TABLE table=test-table [id=874eea9c9921451ca790ae8eb3126247]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:33:08.615134 16942 tablet_service.cc:1467] Processing CreateTablet for tablet c058b842d7c845b0966d26e475fa8a50 (DEFAULT_TABLE table=test-table [id=874eea9c9921451ca790ae8eb3126247]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:33:08.616094 17222 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c058b842d7c845b0966d26e475fa8a50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:33:08.616106 16942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c058b842d7c845b0966d26e475fa8a50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:33:08.616106 17082 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c058b842d7c845b0966d26e475fa8a50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:33:08.623322 17311 tablet_bootstrap.cc:492] T c058b842d7c845b0966d26e475fa8a50 P 67856bddf7404c0c99446a985257bc7e: Bootstrap starting.
I20260929 10:33:08.624220 17313 tablet_bootstrap.cc:492] T c058b842d7c845b0966d26e475fa8a50 P 86dae52b03fd4ebab95bbdb5d4a45d8a: Bootstrap starting.
I20260929 10:33:08.625749 17311 tablet_bootstrap.cc:654] T c058b842d7c845b0966d26e475fa8a50 P 67856bddf7404c0c99446a985257bc7e: Neither blocks nor log segments found. Creating new log.
I20260929 10:33:08.626044 17313 tablet_bootstrap.cc:654] T c058b842d7c845b0966d26e475fa8a50 P 86dae52b03fd4ebab95bbdb5d4a45d8a: Neither blocks nor log segments found. Creating new log.
I20260929 10:33:08.626713 17311 log.cc:824] T c058b842d7c845b0966d26e475fa8a50 P 67856bddf7404c0c99446a985257bc7e: Log is configured to *not* fsync() on all Append() calls
I20260929 10:33:08.626885 17313 log.cc:824] T c058b842d7c845b0966d26e475fa8a50 P 86dae52b03fd4ebab95bbdb5d4a45d8a: Log is configured to *not* fsync() on all Append() calls
I20260929 10:33:08.627586 17312 tablet_bootstrap.cc:492] T c058b842d7c845b0966d26e475fa8a50 P c8405f63549b4e06a48843842f6e1d3c: Bootstrap starting.
I20260929 10:33:08.628712 17311 tablet_bootstrap.cc:492] T c058b842d7c845b0966d26e475fa8a50 P 67856bddf7404c0c99446a985257bc7e: No bootstrap required, opened a new log
I20260929 10:33:08.628875 17313 tablet_bootstrap.cc:492] T c058b842d7c845b0966d26e475fa8a50 P 86dae52b03fd4ebab95bbdb5d4a45d8a: No bootstrap required, opened a new log
I20260929 10:33:08.628944 17311 ts_tablet_manager.cc:1397] T c058b842d7c845b0966d26e475fa8a50 P 67856bddf7404c0c99446a985257bc7e: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260929 10:33:08.629053 17313 ts_tablet_manager.cc:1397] T c058b842d7c845b0966d26e475fa8a50 P 86dae52b03fd4ebab95bbdb5d4a45d8a: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260929 10:33:08.629710 17312 tablet_bootstrap.cc:654] T c058b842d7c845b0966d26e475fa8a50 P c8405f63549b4e06a48843842f6e1d3c: Neither blocks nor log segments found. Creating new log.
I20260929 10:33:08.631058 17312 log.cc:824] T c058b842d7c845b0966d26e475fa8a50 P c8405f63549b4e06a48843842f6e1d3c: Log is configured to *not* fsync() on all Append() calls
I20260929 10:33:08.632357 17311 raft_consensus.cc:348] T c058b842d7c845b0966d26e475fa8a50 P 67856bddf7404c0c99446a985257bc7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86dae52b03fd4ebab95bbdb5d4a45d8a" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34809 } } peers { permanent_uuid: "67856bddf7404c0c99446a985257bc7e" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36029 } } peers { permanent_uuid: "c8405f63549b4e06a48843842f6e1d3c" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 43109 } }
I20260929 10:33:08.632745 17311 raft_consensus.cc:374] T c058b842d7c845b0966d26e475fa8a50 P 67856bddf7404c0c99446a985257bc7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:33:08.632850 17311 raft_consensus.cc:729] T c058b842d7c845b0966d26e475fa8a50 P 67856bddf7404c0c99446a985257bc7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67856bddf7404c0c99446a985257bc7e, State: Initialized, Role: FOLLOWER
I20260929 10:33:08.633035 17312 tablet_bootstrap.cc:492] T c058b842d7c845b0966d26e475fa8a50 P c8405f63549b4e06a48843842f6e1d3c: No bootstrap required, opened a new log
I20260929 10:33:08.632913 17313 raft_consensus.cc:348] T c058b842d7c845b0966d26e475fa8a50 P 86dae52b03fd4ebab95bbdb5d4a45d8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86dae52b03fd4ebab95bbdb5d4a45d8a" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34809 } } peers { permanent_uuid: "67856bddf7404c0c99446a985257bc7e" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36029 } } peers { permanent_uuid: "c8405f63549b4e06a48843842f6e1d3c" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 43109 } }
I20260929 10:33:08.633152 17313 raft_consensus.cc:374] T c058b842d7c845b0966d26e475fa8a50 P 86dae52b03fd4ebab95bbdb5d4a45d8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:33:08.633217 17312 ts_tablet_manager.cc:1397] T c058b842d7c845b0966d26e475fa8a50 P c8405f63549b4e06a48843842f6e1d3c: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260929 10:33:08.633215 17313 raft_consensus.cc:729] T c058b842d7c845b0966d26e475fa8a50 P 86dae52b03fd4ebab95bbdb5d4a45d8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86dae52b03fd4ebab95bbdb5d4a45d8a, State: Initialized, Role: FOLLOWER
I20260929 10:33:08.633355 17311 consensus_queue.cc:260] T c058b842d7c845b0966d26e475fa8a50 P 67856bddf7404c0c99446a985257bc7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86dae52b03fd4ebab95bbdb5d4a45d8a" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34809 } } peers { permanent_uuid: "67856bddf7404c0c99446a985257bc7e" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36029 } } peers { permanent_uuid: "c8405f63549b4e06a48843842f6e1d3c" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 43109 } }
I20260929 10:33:08.633636 17313 consensus_queue.cc:260] T c058b842d7c845b0966d26e475fa8a50 P 86dae52b03fd4ebab95bbdb5d4a45d8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86dae52b03fd4ebab95bbdb5d4a45d8a" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34809 } } peers { permanent_uuid: "67856bddf7404c0c99446a985257bc7e" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36029 } } peers { permanent_uuid: "c8405f63549b4e06a48843842f6e1d3c" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 43109 } }
I20260929 10:33:08.634030 17288 heartbeater.cc:503] Master 127.15.82.126:40081 was elected leader, sending a full tablet report...
I20260929 10:33:08.634438 17311 ts_tablet_manager.cc:1428] T c058b842d7c845b0966d26e475fa8a50 P 67856bddf7404c0c99446a985257bc7e: Time spent starting tablet: real 0.005s user 0.007s sys 0.000s
I20260929 10:33:08.634729 17313 ts_tablet_manager.cc:1428] T c058b842d7c845b0966d26e475fa8a50 P 86dae52b03fd4ebab95bbdb5d4a45d8a: Time spent starting tablet: real 0.006s user 0.002s sys 0.005s
I20260929 10:33:08.635088 17148 heartbeater.cc:503] Master 127.15.82.126:40081 was elected leader, sending a full tablet report...
I20260929 10:33:08.636705 17312 raft_consensus.cc:348] T c058b842d7c845b0966d26e475fa8a50 P c8405f63549b4e06a48843842f6e1d3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86dae52b03fd4ebab95bbdb5d4a45d8a" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34809 } } peers { permanent_uuid: "67856bddf7404c0c99446a985257bc7e" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36029 } } peers { permanent_uuid: "c8405f63549b4e06a48843842f6e1d3c" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 43109 } }
I20260929 10:33:08.636921 17312 raft_consensus.cc:374] T c058b842d7c845b0966d26e475fa8a50 P c8405f63549b4e06a48843842f6e1d3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:33:08.637017 17312 raft_consensus.cc:729] T c058b842d7c845b0966d26e475fa8a50 P c8405f63549b4e06a48843842f6e1d3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8405f63549b4e06a48843842f6e1d3c, State: Initialized, Role: FOLLOWER
I20260929 10:33:08.637440 17312 consensus_queue.cc:260] T c058b842d7c845b0966d26e475fa8a50 P c8405f63549b4e06a48843842f6e1d3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86dae52b03fd4ebab95bbdb5d4a45d8a" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34809 } } peers { permanent_uuid: "67856bddf7404c0c99446a985257bc7e" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36029 } } peers { permanent_uuid: "c8405f63549b4e06a48843842f6e1d3c" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 43109 } }
I20260929 10:33:08.638000 17008 heartbeater.cc:503] Master 127.15.82.126:40081 was elected leader, sending a full tablet report...
I20260929 10:33:08.638347 17312 ts_tablet_manager.cc:1428] T c058b842d7c845b0966d26e475fa8a50 P c8405f63549b4e06a48843842f6e1d3c: Time spent starting tablet: real 0.005s user 0.001s sys 0.005s
I20260929 10:33:08.783213 17319 raft_consensus.cc:482] T c058b842d7c845b0966d26e475fa8a50 P c8405f63549b4e06a48843842f6e1d3c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:33:08.783414 17319 raft_consensus.cc:504] T c058b842d7c845b0966d26e475fa8a50 P c8405f63549b4e06a48843842f6e1d3c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86dae52b03fd4ebab95bbdb5d4a45d8a" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34809 } } peers { permanent_uuid: "67856bddf7404c0c99446a985257bc7e" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36029 } } peers { permanent_uuid: "c8405f63549b4e06a48843842f6e1d3c" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 43109 } }
I20260929 10:33:08.784531 17319 leader_election.cc:288] T c058b842d7c845b0966d26e475fa8a50 P c8405f63549b4e06a48843842f6e1d3c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 86dae52b03fd4ebab95bbdb5d4a45d8a (127.15.82.66:34809), 67856bddf7404c0c99446a985257bc7e (127.15.82.67:36029)
I20260929 10:33:08.788609 17306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:08.785017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.65:54665 (local address 127.15.82.66:34809)
0929 10:33:08.785144 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:08.785148 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:08.785473 (+ 325us) server_negotiation.cc:407] Connection header received
0929 10:33:08.785582 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:08.785585 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:08.785635 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:08.785710 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:33:08.786002 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:08.786572 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:08.787628 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:08.788020 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:08.788146 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:08.788203 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:08.788396 (+ 193us) server_negotiation.cc:299] Negotiation successful
0929 10:33:08.788464 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260929 10:33:08.788618 17307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:08.784924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.65:54027 (local address 127.15.82.67:36029)
0929 10:33:08.785085 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:08.785090 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:08.785338 (+ 248us) server_negotiation.cc:407] Connection header received
0929 10:33:08.785439 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:08.785442 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:08.785493 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:08.785582 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:33:08.785906 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:08.786481 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:08.787629 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:08.788022 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:08.788152 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:08.788203 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:08.788433 (+ 230us) server_negotiation.cc:299] Negotiation successful
0929 10:33:08.788516 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260929 10:33:08.788640 17320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:08.784802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.67:36029 (local address 127.15.82.65:54027)
0929 10:33:08.785233 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:08.785245 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:08.785348 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:08.785610 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:08.785613 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:08.785629 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:33:08.785768 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:08.785786 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:08.786593 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:08.786599 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:08.787503 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:08.787512 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:08.788146 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:08.788149 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:08.788288 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:08.788305 (+ 17us) client_negotiation.cc:769] Sending connection context
0929 10:33:08.788396 (+ 91us) client_negotiation.cc:240] Negotiation successful
0929 10:33:08.788464 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":368,"mutex_wait_us":31,"thread_start_us":156,"threads_started":1}
I20260929 10:33:08.788645 17321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:08.784802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.66:34809 (local address 127.15.82.65:54665)
0929 10:33:08.785346 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:08.785362 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:08.785478 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:08.785743 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:08.785746 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:08.785760 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:33:08.785887 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:08.785893 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:08.786686 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:08.786689 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:08.787501 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:08.787510 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:08.788144 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:08.788147 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:08.788252 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:08.788292 (+ 40us) client_negotiation.cc:769] Sending connection context
0929 10:33:08.788352 (+ 60us) client_negotiation.cc:240] Negotiation successful
0929 10:33:08.788445 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"mutex_wait_us":92,"thread_start_us":112,"threads_started":1}
I20260929 10:33:08.789234 17242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c058b842d7c845b0966d26e475fa8a50" candidate_uuid: "c8405f63549b4e06a48843842f6e1d3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67856bddf7404c0c99446a985257bc7e" is_pre_election: true
I20260929 10:33:08.789465 17101 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c058b842d7c845b0966d26e475fa8a50" candidate_uuid: "c8405f63549b4e06a48843842f6e1d3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86dae52b03fd4ebab95bbdb5d4a45d8a" is_pre_election: true
I20260929 10:33:08.789552 17242 raft_consensus.cc:2454] T c058b842d7c845b0966d26e475fa8a50 P 67856bddf7404c0c99446a985257bc7e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8405f63549b4e06a48843842f6e1d3c in term 0.
I20260929 10:33:08.789759 17101 raft_consensus.cc:2454] T c058b842d7c845b0966d26e475fa8a50 P 86dae52b03fd4ebab95bbdb5d4a45d8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8405f63549b4e06a48843842f6e1d3c in term 0.
I20260929 10:33:08.790052 16896 leader_election.cc:302] T c058b842d7c845b0966d26e475fa8a50 P c8405f63549b4e06a48843842f6e1d3c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67856bddf7404c0c99446a985257bc7e, c8405f63549b4e06a48843842f6e1d3c; no voters:
I20260929 10:33:08.790366 17319 raft_consensus.cc:2781] T c058b842d7c845b0966d26e475fa8a50 P c8405f63549b4e06a48843842f6e1d3c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:33:08.790509 17319 raft_consensus.cc:482] T c058b842d7c845b0966d26e475fa8a50 P c8405f63549b4e06a48843842f6e1d3c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:33:08.790585 17319 raft_consensus.cc:3037] T c058b842d7c845b0966d26e475fa8a50 P c8405f63549b4e06a48843842f6e1d3c [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:33:08.791662 17319 raft_consensus.cc:504] T c058b842d7c845b0966d26e475fa8a50 P c8405f63549b4e06a48843842f6e1d3c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86dae52b03fd4ebab95bbdb5d4a45d8a" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34809 } } peers { permanent_uuid: "67856bddf7404c0c99446a985257bc7e" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36029 } } peers { permanent_uuid: "c8405f63549b4e06a48843842f6e1d3c" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 43109 } }
I20260929 10:33:08.791965 17319 leader_election.cc:288] T c058b842d7c845b0966d26e475fa8a50 P c8405f63549b4e06a48843842f6e1d3c [CANDIDATE]: Term 1 election: Requested vote from peers 86dae52b03fd4ebab95bbdb5d4a45d8a (127.15.82.66:34809), 67856bddf7404c0c99446a985257bc7e (127.15.82.67:36029)
I20260929 10:33:08.792335 17242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c058b842d7c845b0966d26e475fa8a50" candidate_uuid: "c8405f63549b4e06a48843842f6e1d3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67856bddf7404c0c99446a985257bc7e"
I20260929 10:33:08.792424 17101 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c058b842d7c845b0966d26e475fa8a50" candidate_uuid: "c8405f63549b4e06a48843842f6e1d3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86dae52b03fd4ebab95bbdb5d4a45d8a"
I20260929 10:33:08.792469 17242 raft_consensus.cc:3037] T c058b842d7c845b0966d26e475fa8a50 P 67856bddf7404c0c99446a985257bc7e [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:33:08.792539 17101 raft_consensus.cc:3037] T c058b842d7c845b0966d26e475fa8a50 P 86dae52b03fd4ebab95bbdb5d4a45d8a [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:33:08.794205 17101 raft_consensus.cc:2454] T c058b842d7c845b0966d26e475fa8a50 P 86dae52b03fd4ebab95bbdb5d4a45d8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8405f63549b4e06a48843842f6e1d3c in term 1.
I20260929 10:33:08.794205 17242 raft_consensus.cc:2454] T c058b842d7c845b0966d26e475fa8a50 P 67856bddf7404c0c99446a985257bc7e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8405f63549b4e06a48843842f6e1d3c in term 1.
I20260929 10:33:08.794549 16896 leader_election.cc:302] T c058b842d7c845b0966d26e475fa8a50 P c8405f63549b4e06a48843842f6e1d3c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67856bddf7404c0c99446a985257bc7e, c8405f63549b4e06a48843842f6e1d3c; no voters:
I20260929 10:33:08.794744 17319 raft_consensus.cc:2781] T c058b842d7c845b0966d26e475fa8a50 P c8405f63549b4e06a48843842f6e1d3c [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:33:08.795027 17319 raft_consensus.cc:686] T c058b842d7c845b0966d26e475fa8a50 P c8405f63549b4e06a48843842f6e1d3c [term 1 LEADER]: Becoming Leader. State: Replica: c8405f63549b4e06a48843842f6e1d3c, State: Running, Role: LEADER
I20260929 10:33:08.795370 17319 consensus_queue.cc:237] T c058b842d7c845b0966d26e475fa8a50 P c8405f63549b4e06a48843842f6e1d3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86dae52b03fd4ebab95bbdb5d4a45d8a" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34809 } } peers { permanent_uuid: "67856bddf7404c0c99446a985257bc7e" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36029 } } peers { permanent_uuid: "c8405f63549b4e06a48843842f6e1d3c" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 43109 } }
W20260929 10:33:08.797267 17289 tablet.cc:2369] T c058b842d7c845b0966d26e475fa8a50 P 67856bddf7404c0c99446a985257bc7e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 10:33:08.798676 16800 catalog_manager.cc:5495] T c058b842d7c845b0966d26e475fa8a50 P c8405f63549b4e06a48843842f6e1d3c reported cstate change: term changed from 0 to 1, leader changed from <none> to c8405f63549b4e06a48843842f6e1d3c (127.15.82.65). New cstate: current_term: 1 leader_uuid: "c8405f63549b4e06a48843842f6e1d3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86dae52b03fd4ebab95bbdb5d4a45d8a" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "67856bddf7404c0c99446a985257bc7e" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8405f63549b4e06a48843842f6e1d3c" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 43109 } health_report { overall_health: HEALTHY } } }
W20260929 10:33:08.831310 17149 tablet.cc:2369] T c058b842d7c845b0966d26e475fa8a50 P 86dae52b03fd4ebab95bbdb5d4a45d8a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 10:33:08.857120 17310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:08.853699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45730 (local address 127.15.82.65:43109)
0929 10:33:08.853829 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:08.853833 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:08.853920 (+ 87us) server_negotiation.cc:407] Connection header received
0929 10:33:08.854106 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:08.854108 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:08.854161 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:08.854246 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:33:08.854528 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:08.855027 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:08.855835 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:08.856041 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:08.856118 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:08.856391 (+ 273us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:33:08.856780 (+ 389us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:33:08.856865 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:08.856939 (+ 74us) server_negotiation.cc:299] Negotiation successful
0929 10:33:08.856989 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260929 10:33:08.860430 17009 tablet.cc:2369] T c058b842d7c845b0966d26e475fa8a50 P c8405f63549b4e06a48843842f6e1d3c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 10:33:08.862936 17101 raft_consensus.cc:1261] T c058b842d7c845b0966d26e475fa8a50 P 86dae52b03fd4ebab95bbdb5d4a45d8a [term 1 FOLLOWER]: Refusing update from remote peer c8405f63549b4e06a48843842f6e1d3c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:33:08.862949 17242 raft_consensus.cc:1261] T c058b842d7c845b0966d26e475fa8a50 P 67856bddf7404c0c99446a985257bc7e [term 1 FOLLOWER]: Refusing update from remote peer c8405f63549b4e06a48843842f6e1d3c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:33:08.863619 17322 consensus_queue.cc:1035] T c058b842d7c845b0966d26e475fa8a50 P c8405f63549b4e06a48843842f6e1d3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "86dae52b03fd4ebab95bbdb5d4a45d8a" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:33:08.863782 17319 consensus_queue.cc:1035] T c058b842d7c845b0966d26e475fa8a50 P c8405f63549b4e06a48843842f6e1d3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "67856bddf7404c0c99446a985257bc7e" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:33:08.870946 17327 mvcc.cc:205] Tried to move back new op lower bound from 7334617042375323648 to 7334617042108481536. Current Snapshot: MvccSnapshot[applied={T|T < 7334617042375323648}]
I20260929 10:33:08.882068 16800 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46398:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:33:08.882212 16800 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46398:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:33:08.884598 16800 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c8c08b4557844cbbac9edc7f712c305f: Sending DeleteTablet for 3 replicas of tablet c058b842d7c845b0966d26e475fa8a50
I20260929 10:33:08.885285 17082 tablet_service.cc:1514] Processing DeleteTablet for tablet c058b842d7c845b0966d26e475fa8a50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:33:08 UTC) from {username='kudu', principal='kudu/127.15.82.126@KRBTEST.COM'} at 127.0.0.1:37520
I20260929 10:33:08.885304 17222 tablet_service.cc:1514] Processing DeleteTablet for tablet c058b842d7c845b0966d26e475fa8a50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:33:08 UTC) from {username='kudu', principal='kudu/127.15.82.126@KRBTEST.COM'} at 127.0.0.1:56086
I20260929 10:33:08.885304 16942 tablet_service.cc:1514] Processing DeleteTablet for tablet c058b842d7c845b0966d26e475fa8a50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:33:08 UTC) from {username='kudu', principal='kudu/127.15.82.126@KRBTEST.COM'} at 127.0.0.1:45714
I20260929 10:33:08.885680 17334 tablet_replica.cc:330] T c058b842d7c845b0966d26e475fa8a50 P 86dae52b03fd4ebab95bbdb5d4a45d8a: stopping tablet replica
I20260929 10:33:08.885768 17336 tablet_replica.cc:330] T c058b842d7c845b0966d26e475fa8a50 P c8405f63549b4e06a48843842f6e1d3c: stopping tablet replica
I20260929 10:33:08.886059 17334 raft_consensus.cc:2229] T c058b842d7c845b0966d26e475fa8a50 P 86dae52b03fd4ebab95bbdb5d4a45d8a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:33:08.886059 17336 raft_consensus.cc:2229] T c058b842d7c845b0966d26e475fa8a50 P c8405f63549b4e06a48843842f6e1d3c [term 1 LEADER]: Raft consensus shutting down.
I20260929 10:33:08.886379 17336 raft_consensus.cc:2258] T c058b842d7c845b0966d26e475fa8a50 P c8405f63549b4e06a48843842f6e1d3c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:33:08.886452 17334 raft_consensus.cc:2258] T c058b842d7c845b0966d26e475fa8a50 P 86dae52b03fd4ebab95bbdb5d4a45d8a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:33:08.886767 17335 tablet_replica.cc:330] T c058b842d7c845b0966d26e475fa8a50 P 67856bddf7404c0c99446a985257bc7e: stopping tablet replica
I20260929 10:33:08.886724 16805 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46398:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 10:33:08.887008 17335 raft_consensus.cc:2229] T c058b842d7c845b0966d26e475fa8a50 P 67856bddf7404c0c99446a985257bc7e [term 1 FOLLOWER]: Raft consensus shutting down.
W20260929 10:33:08.887137 16805 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:33:08.887292 17335 raft_consensus.cc:2258] T c058b842d7c845b0966d26e475fa8a50 P 67856bddf7404c0c99446a985257bc7e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:33:08.888829 17335 ts_tablet_manager.cc:1905] T c058b842d7c845b0966d26e475fa8a50 P 67856bddf7404c0c99446a985257bc7e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:33:08.888965 17336 ts_tablet_manager.cc:1905] T c058b842d7c845b0966d26e475fa8a50 P c8405f63549b4e06a48843842f6e1d3c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:33:08.891541 17336 ts_tablet_manager.cc:1918] T c058b842d7c845b0966d26e475fa8a50 P c8405f63549b4e06a48843842f6e1d3c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:33:08.891541 17335 ts_tablet_manager.cc:1918] T c058b842d7c845b0966d26e475fa8a50 P 67856bddf7404c0c99446a985257bc7e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:33:08.891638 17335 log.cc:1196] T c058b842d7c845b0966d26e475fa8a50 P 67856bddf7404c0c99446a985257bc7e: Deleting WAL directory at /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/minicluster-data/ts-2/wal/wals/c058b842d7c845b0966d26e475fa8a50
I20260929 10:33:08.891952 17335 ts_tablet_manager.cc:1939] T c058b842d7c845b0966d26e475fa8a50 P 67856bddf7404c0c99446a985257bc7e: Deleting consensus metadata
I20260929 10:33:08.892141 16942 tablet_service.cc:1467] Processing CreateTablet for tablet b964812fc7394af9803f448c79366a68 (DEFAULT_TABLE table=test-table [id=9f1739a065254426932a9502dae8498d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:33:08.892263 17082 tablet_service.cc:1467] Processing CreateTablet for tablet b964812fc7394af9803f448c79366a68 (DEFAULT_TABLE table=test-table [id=9f1739a065254426932a9502dae8498d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:33:08.892405 16942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b964812fc7394af9803f448c79366a68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:33:08.892535 17082 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b964812fc7394af9803f448c79366a68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:33:08.892499 17222 tablet_service.cc:1467] Processing CreateTablet for tablet b964812fc7394af9803f448c79366a68 (DEFAULT_TABLE table=test-table [id=9f1739a065254426932a9502dae8498d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:33:08.893803 17334 ts_tablet_manager.cc:1905] T c058b842d7c845b0966d26e475fa8a50 P 86dae52b03fd4ebab95bbdb5d4a45d8a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:33:08.894637 17313 tablet_bootstrap.cc:492] T b964812fc7394af9803f448c79366a68 P 86dae52b03fd4ebab95bbdb5d4a45d8a: Bootstrap starting.
I20260929 10:33:08.894727 17312 tablet_bootstrap.cc:492] T b964812fc7394af9803f448c79366a68 P c8405f63549b4e06a48843842f6e1d3c: Bootstrap starting.
I20260929 10:33:08.895638 17313 tablet_bootstrap.cc:654] T b964812fc7394af9803f448c79366a68 P 86dae52b03fd4ebab95bbdb5d4a45d8a: Neither blocks nor log segments found. Creating new log.
I20260929 10:33:08.895681 17312 tablet_bootstrap.cc:654] T b964812fc7394af9803f448c79366a68 P c8405f63549b4e06a48843842f6e1d3c: Neither blocks nor log segments found. Creating new log.
I20260929 10:33:08.895767 17336 log.cc:1196] T c058b842d7c845b0966d26e475fa8a50 P c8405f63549b4e06a48843842f6e1d3c: Deleting WAL directory at /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/minicluster-data/ts-0/wal/wals/c058b842d7c845b0966d26e475fa8a50
I20260929 10:33:08.896116 17336 ts_tablet_manager.cc:1939] T c058b842d7c845b0966d26e475fa8a50 P c8405f63549b4e06a48843842f6e1d3c: Deleting consensus metadata
I20260929 10:33:08.896313 17334 ts_tablet_manager.cc:1918] T c058b842d7c845b0966d26e475fa8a50 P 86dae52b03fd4ebab95bbdb5d4a45d8a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:33:08.896404 17334 log.cc:1196] T c058b842d7c845b0966d26e475fa8a50 P 86dae52b03fd4ebab95bbdb5d4a45d8a: Deleting WAL directory at /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/minicluster-data/ts-1/wal/wals/c058b842d7c845b0966d26e475fa8a50
I20260929 10:33:08.896705 17334 ts_tablet_manager.cc:1939] T c058b842d7c845b0966d26e475fa8a50 P 86dae52b03fd4ebab95bbdb5d4a45d8a: Deleting consensus metadata
I20260929 10:33:08.897308 16789 catalog_manager.cc:4841] TS c8405f63549b4e06a48843842f6e1d3c (127.15.82.65:43109): tablet c058b842d7c845b0966d26e475fa8a50 (table test-table [id=874eea9c9921451ca790ae8eb3126247]) successfully deleted
I20260929 10:33:08.897761 16791 catalog_manager.cc:4841] TS 86dae52b03fd4ebab95bbdb5d4a45d8a (127.15.82.66:34809): tablet c058b842d7c845b0966d26e475fa8a50 (table test-table [id=874eea9c9921451ca790ae8eb3126247]) successfully deleted
I20260929 10:33:08.898073 17222 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b964812fc7394af9803f448c79366a68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:33:08.898761 16792 catalog_manager.cc:4841] TS 67856bddf7404c0c99446a985257bc7e (127.15.82.67:36029): tablet c058b842d7c845b0966d26e475fa8a50 (table test-table [id=874eea9c9921451ca790ae8eb3126247]) successfully deleted
I20260929 10:33:08.899154 17312 tablet_bootstrap.cc:492] T b964812fc7394af9803f448c79366a68 P c8405f63549b4e06a48843842f6e1d3c: No bootstrap required, opened a new log
I20260929 10:33:08.899211 17312 ts_tablet_manager.cc:1397] T b964812fc7394af9803f448c79366a68 P c8405f63549b4e06a48843842f6e1d3c: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260929 10:33:08.899621 17312 raft_consensus.cc:348] T b964812fc7394af9803f448c79366a68 P c8405f63549b4e06a48843842f6e1d3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8405f63549b4e06a48843842f6e1d3c" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 43109 } } peers { permanent_uuid: "86dae52b03fd4ebab95bbdb5d4a45d8a" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34809 } } peers { permanent_uuid: "67856bddf7404c0c99446a985257bc7e" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36029 } }
I20260929 10:33:08.899735 17312 raft_consensus.cc:374] T b964812fc7394af9803f448c79366a68 P c8405f63549b4e06a48843842f6e1d3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:33:08.899770 17312 raft_consensus.cc:729] T b964812fc7394af9803f448c79366a68 P c8405f63549b4e06a48843842f6e1d3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8405f63549b4e06a48843842f6e1d3c, State: Initialized, Role: FOLLOWER
I20260929 10:33:08.899893 17312 consensus_queue.cc:260] T b964812fc7394af9803f448c79366a68 P c8405f63549b4e06a48843842f6e1d3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8405f63549b4e06a48843842f6e1d3c" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 43109 } } peers { permanent_uuid: "86dae52b03fd4ebab95bbdb5d4a45d8a" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34809 } } peers { permanent_uuid: "67856bddf7404c0c99446a985257bc7e" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36029 } }
I20260929 10:33:08.900177 17312 ts_tablet_manager.cc:1428] T b964812fc7394af9803f448c79366a68 P c8405f63549b4e06a48843842f6e1d3c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260929 10:33:08.900198 17311 tablet_bootstrap.cc:492] T b964812fc7394af9803f448c79366a68 P 67856bddf7404c0c99446a985257bc7e: Bootstrap starting.
I20260929 10:33:08.901134 17311 tablet_bootstrap.cc:654] T b964812fc7394af9803f448c79366a68 P 67856bddf7404c0c99446a985257bc7e: Neither blocks nor log segments found. Creating new log.
I20260929 10:33:08.901671 17313 tablet_bootstrap.cc:492] T b964812fc7394af9803f448c79366a68 P 86dae52b03fd4ebab95bbdb5d4a45d8a: No bootstrap required, opened a new log
I20260929 10:33:08.901760 17313 ts_tablet_manager.cc:1397] T b964812fc7394af9803f448c79366a68 P 86dae52b03fd4ebab95bbdb5d4a45d8a: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260929 10:33:08.902156 17313 raft_consensus.cc:348] T b964812fc7394af9803f448c79366a68 P 86dae52b03fd4ebab95bbdb5d4a45d8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8405f63549b4e06a48843842f6e1d3c" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 43109 } } peers { permanent_uuid: "86dae52b03fd4ebab95bbdb5d4a45d8a" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34809 } } peers { permanent_uuid: "67856bddf7404c0c99446a985257bc7e" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36029 } }
I20260929 10:33:08.902257 17313 raft_consensus.cc:374] T b964812fc7394af9803f448c79366a68 P 86dae52b03fd4ebab95bbdb5d4a45d8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:33:08.902287 17313 raft_consensus.cc:729] T b964812fc7394af9803f448c79366a68 P 86dae52b03fd4ebab95bbdb5d4a45d8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86dae52b03fd4ebab95bbdb5d4a45d8a, State: Initialized, Role: FOLLOWER
I20260929 10:33:08.902442 17311 tablet_bootstrap.cc:492] T b964812fc7394af9803f448c79366a68 P 67856bddf7404c0c99446a985257bc7e: No bootstrap required, opened a new log
I20260929 10:33:08.902462 17313 consensus_queue.cc:260] T b964812fc7394af9803f448c79366a68 P 86dae52b03fd4ebab95bbdb5d4a45d8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8405f63549b4e06a48843842f6e1d3c" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 43109 } } peers { permanent_uuid: "86dae52b03fd4ebab95bbdb5d4a45d8a" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34809 } } peers { permanent_uuid: "67856bddf7404c0c99446a985257bc7e" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36029 } }
I20260929 10:33:08.902518 17311 ts_tablet_manager.cc:1397] T b964812fc7394af9803f448c79366a68 P 67856bddf7404c0c99446a985257bc7e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 10:33:08.902701 17313 ts_tablet_manager.cc:1428] T b964812fc7394af9803f448c79366a68 P 86dae52b03fd4ebab95bbdb5d4a45d8a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260929 10:33:08.902980 17311 raft_consensus.cc:348] T b964812fc7394af9803f448c79366a68 P 67856bddf7404c0c99446a985257bc7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8405f63549b4e06a48843842f6e1d3c" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 43109 } } peers { permanent_uuid: "86dae52b03fd4ebab95bbdb5d4a45d8a" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34809 } } peers { permanent_uuid: "67856bddf7404c0c99446a985257bc7e" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36029 } }
I20260929 10:33:08.903098 17311 raft_consensus.cc:374] T b964812fc7394af9803f448c79366a68 P 67856bddf7404c0c99446a985257bc7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:33:08.903134 17311 raft_consensus.cc:729] T b964812fc7394af9803f448c79366a68 P 67856bddf7404c0c99446a985257bc7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67856bddf7404c0c99446a985257bc7e, State: Initialized, Role: FOLLOWER
I20260929 10:33:08.903257 17311 consensus_queue.cc:260] T b964812fc7394af9803f448c79366a68 P 67856bddf7404c0c99446a985257bc7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8405f63549b4e06a48843842f6e1d3c" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 43109 } } peers { permanent_uuid: "86dae52b03fd4ebab95bbdb5d4a45d8a" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34809 } } peers { permanent_uuid: "67856bddf7404c0c99446a985257bc7e" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36029 } }
I20260929 10:33:08.903478 17311 ts_tablet_manager.cc:1428] T b964812fc7394af9803f448c79366a68 P 67856bddf7404c0c99446a985257bc7e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260929 10:33:09.047363 17317 raft_consensus.cc:482] T b964812fc7394af9803f448c79366a68 P 67856bddf7404c0c99446a985257bc7e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:33:09.047570 17317 raft_consensus.cc:504] T b964812fc7394af9803f448c79366a68 P 67856bddf7404c0c99446a985257bc7e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8405f63549b4e06a48843842f6e1d3c" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 43109 } } peers { permanent_uuid: "86dae52b03fd4ebab95bbdb5d4a45d8a" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34809 } } peers { permanent_uuid: "67856bddf7404c0c99446a985257bc7e" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36029 } }
I20260929 10:33:09.048815 17317 leader_election.cc:288] T b964812fc7394af9803f448c79366a68 P 67856bddf7404c0c99446a985257bc7e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8405f63549b4e06a48843842f6e1d3c (127.15.82.65:43109), 86dae52b03fd4ebab95bbdb5d4a45d8a (127.15.82.66:34809)
I20260929 10:33:09.053510 17310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:09.049120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.67:39701 (local address 127.15.82.65:43109)
0929 10:33:09.049281 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:09.049286 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:09.049329 (+ 43us) server_negotiation.cc:407] Connection header received
0929 10:33:09.049392 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:09.049394 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:09.049446 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:09.049547 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:33:09.049890 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:09.050690 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:09.052222 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:09.052819 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:09.052940 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:09.052992 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:09.053300 (+ 308us) server_negotiation.cc:299] Negotiation successful
0929 10:33:09.053366 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260929 10:33:09.053500 17306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:09.049313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.67:46199 (local address 127.15.82.66:34809)
0929 10:33:09.049440 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:09.049444 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:09.049608 (+ 164us) server_negotiation.cc:407] Connection header received
0929 10:33:09.049723 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:09.049725 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:09.049761 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:09.049819 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:33:09.050158 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:09.050896 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:09.052249 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:09.052819 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:09.052940 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:09.052997 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:09.053300 (+ 303us) server_negotiation.cc:299] Negotiation successful
0929 10:33:09.053366 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260929 10:33:09.053546 17340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:09.049112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.66:34809 (local address 127.15.82.67:46199)
0929 10:33:09.049480 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:09.049491 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:09.049620 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:09.049861 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:09.049867 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:09.049886 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:33:09.050051 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:09.050056 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:09.050984 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:09.050987 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:09.052140 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:09.052151 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:09.052950 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:09.052954 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:09.053108 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:09.053187 (+ 79us) client_negotiation.cc:769] Sending connection context
0929 10:33:09.053266 (+ 79us) client_negotiation.cc:240] Negotiation successful
0929 10:33:09.053363 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"mutex_wait_us":95,"spinlock_wait_cycles":1792,"thread_start_us":104,"threads_started":1}
I20260929 10:33:09.053522 17181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:09.049034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.65:43109 (local address 127.15.82.67:39701)
0929 10:33:09.049149 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:09.049169 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:09.049267 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:09.049597 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:09.049600 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:09.049620 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:33:09.049760 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:09.049765 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:09.050795 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:09.050798 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:09.052093 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:09.052104 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:09.052939 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:09.052944 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:09.053129 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:09.053188 (+ 59us) client_negotiation.cc:769] Sending connection context
0929 10:33:09.053266 (+ 78us) client_negotiation.cc:240] Negotiation successful
0929 10:33:09.053363 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31,"mutex_wait_us":55,"spinlock_wait_cycles":1280}
I20260929 10:33:09.054056 17101 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b964812fc7394af9803f448c79366a68" candidate_uuid: "67856bddf7404c0c99446a985257bc7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86dae52b03fd4ebab95bbdb5d4a45d8a" is_pre_election: true
I20260929 10:33:09.054126 16962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b964812fc7394af9803f448c79366a68" candidate_uuid: "67856bddf7404c0c99446a985257bc7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8405f63549b4e06a48843842f6e1d3c" is_pre_election: true
I20260929 10:33:09.054219 17101 raft_consensus.cc:2454] T b964812fc7394af9803f448c79366a68 P 86dae52b03fd4ebab95bbdb5d4a45d8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67856bddf7404c0c99446a985257bc7e in term 0.
I20260929 10:33:09.054380 16962 raft_consensus.cc:2454] T b964812fc7394af9803f448c79366a68 P c8405f63549b4e06a48843842f6e1d3c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67856bddf7404c0c99446a985257bc7e in term 0.
I20260929 10:33:09.054600 17175 leader_election.cc:302] T b964812fc7394af9803f448c79366a68 P 67856bddf7404c0c99446a985257bc7e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67856bddf7404c0c99446a985257bc7e, 86dae52b03fd4ebab95bbdb5d4a45d8a; no voters:
I20260929 10:33:09.054867 17317 raft_consensus.cc:2781] T b964812fc7394af9803f448c79366a68 P 67856bddf7404c0c99446a985257bc7e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:33:09.054941 17317 raft_consensus.cc:482] T b964812fc7394af9803f448c79366a68 P 67856bddf7404c0c99446a985257bc7e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:33:09.054996 17317 raft_consensus.cc:3037] T b964812fc7394af9803f448c79366a68 P 67856bddf7404c0c99446a985257bc7e [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:33:09.056125 17317 raft_consensus.cc:504] T b964812fc7394af9803f448c79366a68 P 67856bddf7404c0c99446a985257bc7e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8405f63549b4e06a48843842f6e1d3c" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 43109 } } peers { permanent_uuid: "86dae52b03fd4ebab95bbdb5d4a45d8a" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34809 } } peers { permanent_uuid: "67856bddf7404c0c99446a985257bc7e" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36029 } }
I20260929 10:33:09.056463 17317 leader_election.cc:288] T b964812fc7394af9803f448c79366a68 P 67856bddf7404c0c99446a985257bc7e [CANDIDATE]: Term 1 election: Requested vote from peers c8405f63549b4e06a48843842f6e1d3c (127.15.82.65:43109), 86dae52b03fd4ebab95bbdb5d4a45d8a (127.15.82.66:34809)
I20260929 10:33:09.056823 17101 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b964812fc7394af9803f448c79366a68" candidate_uuid: "67856bddf7404c0c99446a985257bc7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86dae52b03fd4ebab95bbdb5d4a45d8a"
I20260929 10:33:09.056823 16962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b964812fc7394af9803f448c79366a68" candidate_uuid: "67856bddf7404c0c99446a985257bc7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8405f63549b4e06a48843842f6e1d3c"
I20260929 10:33:09.056941 17101 raft_consensus.cc:3037] T b964812fc7394af9803f448c79366a68 P 86dae52b03fd4ebab95bbdb5d4a45d8a [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:33:09.056941 16962 raft_consensus.cc:3037] T b964812fc7394af9803f448c79366a68 P c8405f63549b4e06a48843842f6e1d3c [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:33:09.058061 17101 raft_consensus.cc:2454] T b964812fc7394af9803f448c79366a68 P 86dae52b03fd4ebab95bbdb5d4a45d8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67856bddf7404c0c99446a985257bc7e in term 1.
I20260929 10:33:09.058079 16962 raft_consensus.cc:2454] T b964812fc7394af9803f448c79366a68 P c8405f63549b4e06a48843842f6e1d3c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67856bddf7404c0c99446a985257bc7e in term 1.
I20260929 10:33:09.058336 17173 leader_election.cc:302] T b964812fc7394af9803f448c79366a68 P 67856bddf7404c0c99446a985257bc7e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67856bddf7404c0c99446a985257bc7e, c8405f63549b4e06a48843842f6e1d3c; no voters:
I20260929 10:33:09.058534 17317 raft_consensus.cc:2781] T b964812fc7394af9803f448c79366a68 P 67856bddf7404c0c99446a985257bc7e [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:33:09.058735 17317 raft_consensus.cc:686] T b964812fc7394af9803f448c79366a68 P 67856bddf7404c0c99446a985257bc7e [term 1 LEADER]: Becoming Leader. State: Replica: 67856bddf7404c0c99446a985257bc7e, State: Running, Role: LEADER
I20260929 10:33:09.059013 17317 consensus_queue.cc:237] T b964812fc7394af9803f448c79366a68 P 67856bddf7404c0c99446a985257bc7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8405f63549b4e06a48843842f6e1d3c" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 43109 } } peers { permanent_uuid: "86dae52b03fd4ebab95bbdb5d4a45d8a" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34809 } } peers { permanent_uuid: "67856bddf7404c0c99446a985257bc7e" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36029 } }
I20260929 10:33:09.061189 16800 catalog_manager.cc:5495] T b964812fc7394af9803f448c79366a68 P 67856bddf7404c0c99446a985257bc7e reported cstate change: term changed from 0 to 1, leader changed from <none> to 67856bddf7404c0c99446a985257bc7e (127.15.82.67). New cstate: current_term: 1 leader_uuid: "67856bddf7404c0c99446a985257bc7e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8405f63549b4e06a48843842f6e1d3c" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 43109 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "86dae52b03fd4ebab95bbdb5d4a45d8a" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "67856bddf7404c0c99446a985257bc7e" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36029 } health_report { overall_health: HEALTHY } } }
Sep 29 10:33:09 dist-test-slave-vg89 krb5kdc[16760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790677987, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.126@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:33:09.124614 17348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:09.116014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:40081 (local address 127.0.0.1:46406)
0929 10:33:09.116263 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:09.116279 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:09.116421 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:09.116696 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:09.116699 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:09.117083 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:09.117168 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:09.117174 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:09.117953 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:09.117955 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:09.118608 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:09.118633 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:09.119065 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:09.119068 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:09.119199 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:09.119806 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:09.119819 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:09.121382 (+ 1563us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:09.123068 (+ 1686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:09.123078 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:09.123085 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:09.123381 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:09.123635 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:09.123638 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:09.123641 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:09.123736 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:09.124099 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:09.124106 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:09.124222 (+ 116us) client_negotiation.cc:769] Sending connection context
0929 10:33:09.124316 (+ 94us) client_negotiation.cc:240] Negotiation successful
0929 10:33:09.124450 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":38,"thread_start_us":75,"threads_started":1}
I20260929 10:33:09.124634 17349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:09.116084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46406 (local address 127.15.82.126:40081)
0929 10:33:09.116266 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:09.116273 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:09.116364 (+ 91us) server_negotiation.cc:407] Connection header received
0929 10:33:09.116542 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:09.116545 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:09.116590 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:09.116681 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:09.117283 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:09.117811 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:09.118739 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:09.118955 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:09.119067 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:09.121523 (+ 2456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:09.121526 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:09.121528 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:09.121550 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:09.122956 (+ 1406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:09.123472 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:09.123475 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:09.123477 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:09.123552 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:09.123821 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:09.123823 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:09.123824 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:09.123996 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:09.124068 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:09.124378 (+ 310us) server_negotiation.cc:299] Negotiation successful
0929 10:33:09.124503 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":139,"thread_start_us":72,"threads_started":1}
I20260929 10:33:09.128494 16800 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.15.82.126@KRBTEST.COM'} at 127.0.0.1:46406:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260929 10:33:09.129168 16800 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:33:09.133672 16942 tablet_service.cc:1467] Processing CreateTablet for tablet 4867362bfccc40f6867eeb1b3ece982f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6814d98130fc433587472e595072de17]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 10:33:09.133730 17082 tablet_service.cc:1467] Processing CreateTablet for tablet 4867362bfccc40f6867eeb1b3ece982f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6814d98130fc433587472e595072de17]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 10:33:09.133860 17222 tablet_service.cc:1467] Processing CreateTablet for tablet 4867362bfccc40f6867eeb1b3ece982f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6814d98130fc433587472e595072de17]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 10:33:09.134100 16942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4867362bfccc40f6867eeb1b3ece982f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:33:09.134141 17082 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4867362bfccc40f6867eeb1b3ece982f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:33:09.134186 17222 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4867362bfccc40f6867eeb1b3ece982f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:33:09.137087 17312 tablet_bootstrap.cc:492] T 4867362bfccc40f6867eeb1b3ece982f P c8405f63549b4e06a48843842f6e1d3c: Bootstrap starting.
I20260929 10:33:09.137545 17311 tablet_bootstrap.cc:492] T 4867362bfccc40f6867eeb1b3ece982f P 67856bddf7404c0c99446a985257bc7e: Bootstrap starting.
I20260929 10:33:09.137852 17312 tablet_bootstrap.cc:654] T 4867362bfccc40f6867eeb1b3ece982f P c8405f63549b4e06a48843842f6e1d3c: Neither blocks nor log segments found. Creating new log.
I20260929 10:33:09.138338 17311 tablet_bootstrap.cc:654] T 4867362bfccc40f6867eeb1b3ece982f P 67856bddf7404c0c99446a985257bc7e: Neither blocks nor log segments found. Creating new log.
I20260929 10:33:09.138525 17313 tablet_bootstrap.cc:492] T 4867362bfccc40f6867eeb1b3ece982f P 86dae52b03fd4ebab95bbdb5d4a45d8a: Bootstrap starting.
I20260929 10:33:09.139075 17312 tablet_bootstrap.cc:492] T 4867362bfccc40f6867eeb1b3ece982f P c8405f63549b4e06a48843842f6e1d3c: No bootstrap required, opened a new log
I20260929 10:33:09.139175 17312 ts_tablet_manager.cc:1397] T 4867362bfccc40f6867eeb1b3ece982f P c8405f63549b4e06a48843842f6e1d3c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 10:33:09.139540 17313 tablet_bootstrap.cc:654] T 4867362bfccc40f6867eeb1b3ece982f P 86dae52b03fd4ebab95bbdb5d4a45d8a: Neither blocks nor log segments found. Creating new log.
I20260929 10:33:09.139590 17311 tablet_bootstrap.cc:492] T 4867362bfccc40f6867eeb1b3ece982f P 67856bddf7404c0c99446a985257bc7e: No bootstrap required, opened a new log
I20260929 10:33:09.139657 17311 ts_tablet_manager.cc:1397] T 4867362bfccc40f6867eeb1b3ece982f P 67856bddf7404c0c99446a985257bc7e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 10:33:09.139623 17312 raft_consensus.cc:348] T 4867362bfccc40f6867eeb1b3ece982f P c8405f63549b4e06a48843842f6e1d3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8405f63549b4e06a48843842f6e1d3c" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 43109 } } peers { permanent_uuid: "86dae52b03fd4ebab95bbdb5d4a45d8a" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34809 } } peers { permanent_uuid: "67856bddf7404c0c99446a985257bc7e" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36029 } }
I20260929 10:33:09.139750 17312 raft_consensus.cc:374] T 4867362bfccc40f6867eeb1b3ece982f P c8405f63549b4e06a48843842f6e1d3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:33:09.139792 17312 raft_consensus.cc:729] T 4867362bfccc40f6867eeb1b3ece982f P c8405f63549b4e06a48843842f6e1d3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8405f63549b4e06a48843842f6e1d3c, State: Initialized, Role: FOLLOWER
I20260929 10:33:09.139928 17312 consensus_queue.cc:260] T 4867362bfccc40f6867eeb1b3ece982f P c8405f63549b4e06a48843842f6e1d3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8405f63549b4e06a48843842f6e1d3c" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 43109 } } peers { permanent_uuid: "86dae52b03fd4ebab95bbdb5d4a45d8a" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34809 } } peers { permanent_uuid: "67856bddf7404c0c99446a985257bc7e" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36029 } }
I20260929 10:33:09.140214 17312 ts_tablet_manager.cc:1428] T 4867362bfccc40f6867eeb1b3ece982f P c8405f63549b4e06a48843842f6e1d3c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260929 10:33:09.140208 17311 raft_consensus.cc:348] T 4867362bfccc40f6867eeb1b3ece982f P 67856bddf7404c0c99446a985257bc7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8405f63549b4e06a48843842f6e1d3c" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 43109 } } peers { permanent_uuid: "86dae52b03fd4ebab95bbdb5d4a45d8a" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34809 } } peers { permanent_uuid: "67856bddf7404c0c99446a985257bc7e" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36029 } }
I20260929 10:33:09.140288 17311 raft_consensus.cc:374] T 4867362bfccc40f6867eeb1b3ece982f P 67856bddf7404c0c99446a985257bc7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:33:09.140338 17311 raft_consensus.cc:729] T 4867362bfccc40f6867eeb1b3ece982f P 67856bddf7404c0c99446a985257bc7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67856bddf7404c0c99446a985257bc7e, State: Initialized, Role: FOLLOWER
I20260929 10:33:09.140432 17322 tablet_replica.cc:439] T 4867362bfccc40f6867eeb1b3ece982f P c8405f63549b4e06a48843842f6e1d3c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8405f63549b4e06a48843842f6e1d3c" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 43109 } } peers { permanent_uuid: "86dae52b03fd4ebab95bbdb5d4a45d8a" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34809 } } peers { permanent_uuid: "67856bddf7404c0c99446a985257bc7e" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36029 } } }
I20260929 10:33:09.140455 17311 consensus_queue.cc:260] T 4867362bfccc40f6867eeb1b3ece982f P 67856bddf7404c0c99446a985257bc7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8405f63549b4e06a48843842f6e1d3c" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 43109 } } peers { permanent_uuid: "86dae52b03fd4ebab95bbdb5d4a45d8a" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34809 } } peers { permanent_uuid: "67856bddf7404c0c99446a985257bc7e" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36029 } }
I20260929 10:33:09.140591 17322 tablet_replica.cc:442] T 4867362bfccc40f6867eeb1b3ece982f P c8405f63549b4e06a48843842f6e1d3c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 10:33:09.140738 17311 ts_tablet_manager.cc:1428] T 4867362bfccc40f6867eeb1b3ece982f P 67856bddf7404c0c99446a985257bc7e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260929 10:33:09.140949 17317 tablet_replica.cc:439] T 4867362bfccc40f6867eeb1b3ece982f P 67856bddf7404c0c99446a985257bc7e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8405f63549b4e06a48843842f6e1d3c" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 43109 } } peers { permanent_uuid: "86dae52b03fd4ebab95bbdb5d4a45d8a" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34809 } } peers { permanent_uuid: "67856bddf7404c0c99446a985257bc7e" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36029 } } }
I20260929 10:33:09.141076 17317 tablet_replica.cc:442] T 4867362bfccc40f6867eeb1b3ece982f P 67856bddf7404c0c99446a985257bc7e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 10:33:09.142102 17313 tablet_bootstrap.cc:492] T 4867362bfccc40f6867eeb1b3ece982f P 86dae52b03fd4ebab95bbdb5d4a45d8a: No bootstrap required, opened a new log
I20260929 10:33:09.142200 17313 ts_tablet_manager.cc:1397] T 4867362bfccc40f6867eeb1b3ece982f P 86dae52b03fd4ebab95bbdb5d4a45d8a: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260929 10:33:09.142740 17313 raft_consensus.cc:348] T 4867362bfccc40f6867eeb1b3ece982f P 86dae52b03fd4ebab95bbdb5d4a45d8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8405f63549b4e06a48843842f6e1d3c" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 43109 } } peers { permanent_uuid: "86dae52b03fd4ebab95bbdb5d4a45d8a" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34809 } } peers { permanent_uuid: "67856bddf7404c0c99446a985257bc7e" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36029 } }
I20260929 10:33:09.142863 17313 raft_consensus.cc:374] T 4867362bfccc40f6867eeb1b3ece982f P 86dae52b03fd4ebab95bbdb5d4a45d8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:33:09.142910 17313 raft_consensus.cc:729] T 4867362bfccc40f6867eeb1b3ece982f P 86dae52b03fd4ebab95bbdb5d4a45d8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86dae52b03fd4ebab95bbdb5d4a45d8a, State: Initialized, Role: FOLLOWER
I20260929 10:33:09.143034 17313 consensus_queue.cc:260] T 4867362bfccc40f6867eeb1b3ece982f P 86dae52b03fd4ebab95bbdb5d4a45d8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8405f63549b4e06a48843842f6e1d3c" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 43109 } } peers { permanent_uuid: "86dae52b03fd4ebab95bbdb5d4a45d8a" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34809 } } peers { permanent_uuid: "67856bddf7404c0c99446a985257bc7e" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36029 } }
I20260929 10:33:09.143361 17313 ts_tablet_manager.cc:1428] T 4867362bfccc40f6867eeb1b3ece982f P 86dae52b03fd4ebab95bbdb5d4a45d8a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260929 10:33:09.143306 17318 tablet_replica.cc:439] T 4867362bfccc40f6867eeb1b3ece982f P 86dae52b03fd4ebab95bbdb5d4a45d8a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8405f63549b4e06a48843842f6e1d3c" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 43109 } } peers { permanent_uuid: "86dae52b03fd4ebab95bbdb5d4a45d8a" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34809 } } peers { permanent_uuid: "67856bddf7404c0c99446a985257bc7e" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36029 } } }
I20260929 10:33:09.143465 17318 tablet_replica.cc:442] T 4867362bfccc40f6867eeb1b3ece982f P 86dae52b03fd4ebab95bbdb5d4a45d8a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 10:33:09.204820 17318 raft_consensus.cc:482] T 4867362bfccc40f6867eeb1b3ece982f P 86dae52b03fd4ebab95bbdb5d4a45d8a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:33:09.204983 17318 raft_consensus.cc:504] T 4867362bfccc40f6867eeb1b3ece982f P 86dae52b03fd4ebab95bbdb5d4a45d8a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8405f63549b4e06a48843842f6e1d3c" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 43109 } } peers { permanent_uuid: "86dae52b03fd4ebab95bbdb5d4a45d8a" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34809 } } peers { permanent_uuid: "67856bddf7404c0c99446a985257bc7e" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36029 } }
I20260929 10:33:09.206094 17318 leader_election.cc:288] T 4867362bfccc40f6867eeb1b3ece982f P 86dae52b03fd4ebab95bbdb5d4a45d8a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8405f63549b4e06a48843842f6e1d3c (127.15.82.65:43109), 67856bddf7404c0c99446a985257bc7e (127.15.82.67:36029)
I20260929 10:33:09.211565 17310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:09.206458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:34633 (local address 127.15.82.65:43109)
0929 10:33:09.206615 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:09.206620 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:09.206843 (+ 223us) server_negotiation.cc:407] Connection header received
0929 10:33:09.207081 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:09.207085 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:09.207142 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:09.207213 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:33:09.207465 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:09.208258 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:09.210177 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:09.210819 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:09.210945 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:09.210997 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:09.211356 (+ 359us) server_negotiation.cc:299] Negotiation successful
0929 10:33:09.211436 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260929 10:33:09.211565 17307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:09.206516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:35049 (local address 127.15.82.67:36029)
0929 10:33:09.206650 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:09.206654 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:09.206826 (+ 172us) server_negotiation.cc:407] Connection header received
0929 10:33:09.206995 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:09.207000 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:09.207071 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:09.207150 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:33:09.207465 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:09.208252 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:09.210177 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:09.210819 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:09.210945 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:09.210997 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:09.211356 (+ 359us) server_negotiation.cc:299] Negotiation successful
0929 10:33:09.211435 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260929 10:33:09.211593 17353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:09.206320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.65:43109 (local address 127.15.82.66:34633)
0929 10:33:09.206748 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:09.206759 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:09.206901 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:09.207234 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:09.207236 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:09.207247 (+ 11us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:33:09.207361 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:09.207366 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:09.208370 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:09.208374 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:09.210044 (+ 1670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:09.210055 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:09.210947 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:09.210951 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:09.211165 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:09.211246 (+ 81us) client_negotiation.cc:769] Sending connection context
0929 10:33:09.211326 (+ 80us) client_negotiation.cc:240] Negotiation successful
0929 10:33:09.211413 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"mutex_wait_us":300,"spinlock_wait_cycles":4352,"thread_start_us":117,"threads_started":1}
I20260929 10:33:09.211606 17354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:09.206321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.67:36029 (local address 127.15.82.66:35049)
0929 10:33:09.206731 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:09.206743 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:09.206891 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:09.207165 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:09.207168 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:09.207181 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:33:09.207347 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:09.207354 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:09.208370 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:09.208374 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:09.210045 (+ 1671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:09.210055 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:09.210947 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:09.210951 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:09.211169 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:09.211250 (+ 81us) client_negotiation.cc:769] Sending connection context
0929 10:33:09.211326 (+ 76us) client_negotiation.cc:240] Negotiation successful
0929 10:33:09.211440 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"mutex_wait_us":318,"spinlock_wait_cycles":3968,"thread_start_us":142,"threads_started":1}
I20260929 10:33:09.212196 16962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4867362bfccc40f6867eeb1b3ece982f" candidate_uuid: "86dae52b03fd4ebab95bbdb5d4a45d8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8405f63549b4e06a48843842f6e1d3c" is_pre_election: true
I20260929 10:33:09.212369 16962 raft_consensus.cc:2454] T 4867362bfccc40f6867eeb1b3ece982f P c8405f63549b4e06a48843842f6e1d3c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86dae52b03fd4ebab95bbdb5d4a45d8a in term 0.
I20260929 10:33:09.212350 17242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4867362bfccc40f6867eeb1b3ece982f" candidate_uuid: "86dae52b03fd4ebab95bbdb5d4a45d8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67856bddf7404c0c99446a985257bc7e" is_pre_election: true
I20260929 10:33:09.212486 17242 raft_consensus.cc:2454] T 4867362bfccc40f6867eeb1b3ece982f P 67856bddf7404c0c99446a985257bc7e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86dae52b03fd4ebab95bbdb5d4a45d8a in term 0.
I20260929 10:33:09.212863 17036 leader_election.cc:302] T 4867362bfccc40f6867eeb1b3ece982f P 86dae52b03fd4ebab95bbdb5d4a45d8a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67856bddf7404c0c99446a985257bc7e, 86dae52b03fd4ebab95bbdb5d4a45d8a; no voters:
I20260929 10:33:09.213164 17318 raft_consensus.cc:2781] T 4867362bfccc40f6867eeb1b3ece982f P 86dae52b03fd4ebab95bbdb5d4a45d8a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:33:09.213225 17318 raft_consensus.cc:482] T 4867362bfccc40f6867eeb1b3ece982f P 86dae52b03fd4ebab95bbdb5d4a45d8a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:33:09.213274 17318 raft_consensus.cc:3037] T 4867362bfccc40f6867eeb1b3ece982f P 86dae52b03fd4ebab95bbdb5d4a45d8a [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:33:09.214286 17318 raft_consensus.cc:504] T 4867362bfccc40f6867eeb1b3ece982f P 86dae52b03fd4ebab95bbdb5d4a45d8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8405f63549b4e06a48843842f6e1d3c" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 43109 } } peers { permanent_uuid: "86dae52b03fd4ebab95bbdb5d4a45d8a" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34809 } } peers { permanent_uuid: "67856bddf7404c0c99446a985257bc7e" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36029 } }
I20260929 10:33:09.214658 17318 leader_election.cc:288] T 4867362bfccc40f6867eeb1b3ece982f P 86dae52b03fd4ebab95bbdb5d4a45d8a [CANDIDATE]: Term 1 election: Requested vote from peers c8405f63549b4e06a48843842f6e1d3c (127.15.82.65:43109), 67856bddf7404c0c99446a985257bc7e (127.15.82.67:36029)
I20260929 10:33:09.215003 17242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4867362bfccc40f6867eeb1b3ece982f" candidate_uuid: "86dae52b03fd4ebab95bbdb5d4a45d8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67856bddf7404c0c99446a985257bc7e"
I20260929 10:33:09.215040 16962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4867362bfccc40f6867eeb1b3ece982f" candidate_uuid: "86dae52b03fd4ebab95bbdb5d4a45d8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8405f63549b4e06a48843842f6e1d3c"
I20260929 10:33:09.215118 17242 raft_consensus.cc:3037] T 4867362bfccc40f6867eeb1b3ece982f P 67856bddf7404c0c99446a985257bc7e [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:33:09.215161 16962 raft_consensus.cc:3037] T 4867362bfccc40f6867eeb1b3ece982f P c8405f63549b4e06a48843842f6e1d3c [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:33:09.216024 16962 raft_consensus.cc:2454] T 4867362bfccc40f6867eeb1b3ece982f P c8405f63549b4e06a48843842f6e1d3c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86dae52b03fd4ebab95bbdb5d4a45d8a in term 1.
I20260929 10:33:09.216029 17242 raft_consensus.cc:2454] T 4867362bfccc40f6867eeb1b3ece982f P 67856bddf7404c0c99446a985257bc7e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86dae52b03fd4ebab95bbdb5d4a45d8a in term 1.
I20260929 10:33:09.216306 17036 leader_election.cc:302] T 4867362bfccc40f6867eeb1b3ece982f P 86dae52b03fd4ebab95bbdb5d4a45d8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67856bddf7404c0c99446a985257bc7e, 86dae52b03fd4ebab95bbdb5d4a45d8a; no voters:
I20260929 10:33:09.216491 17318 raft_consensus.cc:2781] T 4867362bfccc40f6867eeb1b3ece982f P 86dae52b03fd4ebab95bbdb5d4a45d8a [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:33:09.216706 17318 raft_consensus.cc:686] T 4867362bfccc40f6867eeb1b3ece982f P 86dae52b03fd4ebab95bbdb5d4a45d8a [term 1 LEADER]: Becoming Leader. State: Replica: 86dae52b03fd4ebab95bbdb5d4a45d8a, State: Running, Role: LEADER
I20260929 10:33:09.217160 17318 consensus_queue.cc:237] T 4867362bfccc40f6867eeb1b3ece982f P 86dae52b03fd4ebab95bbdb5d4a45d8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8405f63549b4e06a48843842f6e1d3c" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 43109 } } peers { permanent_uuid: "86dae52b03fd4ebab95bbdb5d4a45d8a" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34809 } } peers { permanent_uuid: "67856bddf7404c0c99446a985257bc7e" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36029 } }
I20260929 10:33:09.218434 17355 tablet_replica.cc:439] T 4867362bfccc40f6867eeb1b3ece982f P 86dae52b03fd4ebab95bbdb5d4a45d8a: TxnStatusTablet state changed. Reason: New leader 86dae52b03fd4ebab95bbdb5d4a45d8a. Latest consensus state: current_term: 1 leader_uuid: "86dae52b03fd4ebab95bbdb5d4a45d8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8405f63549b4e06a48843842f6e1d3c" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 43109 } } peers { permanent_uuid: "86dae52b03fd4ebab95bbdb5d4a45d8a" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34809 } } peers { permanent_uuid: "67856bddf7404c0c99446a985257bc7e" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36029 } } }
I20260929 10:33:09.218556 17355 tablet_replica.cc:442] T 4867362bfccc40f6867eeb1b3ece982f P 86dae52b03fd4ebab95bbdb5d4a45d8a: This TxnStatusTablet replica's current role is: LEADER
I20260929 10:33:09.218814 17357 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260929 10:33:09.218966 17357 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260929 10:33:09.219683 16805 catalog_manager.cc:5495] T 4867362bfccc40f6867eeb1b3ece982f P 86dae52b03fd4ebab95bbdb5d4a45d8a reported cstate change: term changed from 0 to 1, leader changed from <none> to 86dae52b03fd4ebab95bbdb5d4a45d8a (127.15.82.66). New cstate: current_term: 1 leader_uuid: "86dae52b03fd4ebab95bbdb5d4a45d8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8405f63549b4e06a48843842f6e1d3c" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 43109 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "86dae52b03fd4ebab95bbdb5d4a45d8a" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34809 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "67856bddf7404c0c99446a985257bc7e" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36029 } health_report { overall_health: UNKNOWN } } }
I20260929 10:33:09.241408 17306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:09.237627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37526 (local address 127.15.82.66:34809)
0929 10:33:09.237740 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:09.237743 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:09.237756 (+ 13us) server_negotiation.cc:407] Connection header received
0929 10:33:09.237927 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:09.237929 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:09.237980 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:09.238058 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:33:09.238318 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:09.238860 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:09.240089 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:09.240291 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:09.240363 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:09.240680 (+ 317us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:33:09.241045 (+ 365us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:33:09.241125 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:09.241248 (+ 123us) server_negotiation.cc:299] Negotiation successful
0929 10:33:09.241291 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260929 10:33:09.241451 17348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:09.237492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.66:34809 (local address 127.0.0.1:37526)
0929 10:33:09.237674 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:09.237691 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:09.237793 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:09.238099 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:09.238102 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:09.238110 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 10:33:09.238218 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:09.238223 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:09.238971 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:09.238975 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:09.239949 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:09.239960 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:09.240399 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:09.240403 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:09.240498 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:09.240549 (+ 51us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 10:33:09.241140 (+ 591us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 10:33:09.241155 (+ 15us) client_negotiation.cc:769] Sending connection context
0929 10:33:09.241228 (+ 73us) client_negotiation.cc:240] Negotiation successful
0929 10:33:09.241313 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260929 10:33:09.244154 17242 raft_consensus.cc:1261] T 4867362bfccc40f6867eeb1b3ece982f P 67856bddf7404c0c99446a985257bc7e [term 1 FOLLOWER]: Refusing update from remote peer 86dae52b03fd4ebab95bbdb5d4a45d8a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:33:09.244311 16962 raft_consensus.cc:1261] T 4867362bfccc40f6867eeb1b3ece982f P c8405f63549b4e06a48843842f6e1d3c [term 1 FOLLOWER]: Refusing update from remote peer 86dae52b03fd4ebab95bbdb5d4a45d8a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:33:09.244620 17318 consensus_queue.cc:1035] T 4867362bfccc40f6867eeb1b3ece982f P 86dae52b03fd4ebab95bbdb5d4a45d8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "67856bddf7404c0c99446a985257bc7e" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:33:09.244820 17355 consensus_queue.cc:1035] T 4867362bfccc40f6867eeb1b3ece982f P 86dae52b03fd4ebab95bbdb5d4a45d8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8405f63549b4e06a48843842f6e1d3c" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 43109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:33:09.246846 17322 tablet_replica.cc:439] T 4867362bfccc40f6867eeb1b3ece982f P c8405f63549b4e06a48843842f6e1d3c: TxnStatusTablet state changed. Reason: New leader 86dae52b03fd4ebab95bbdb5d4a45d8a. Latest consensus state: current_term: 1 leader_uuid: "86dae52b03fd4ebab95bbdb5d4a45d8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8405f63549b4e06a48843842f6e1d3c" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 43109 } } peers { permanent_uuid: "86dae52b03fd4ebab95bbdb5d4a45d8a" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34809 } } peers { permanent_uuid: "67856bddf7404c0c99446a985257bc7e" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36029 } } }
I20260929 10:33:09.246964 17322 tablet_replica.cc:442] T 4867362bfccc40f6867eeb1b3ece982f P c8405f63549b4e06a48843842f6e1d3c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 10:33:09.247637 17317 tablet_replica.cc:439] T 4867362bfccc40f6867eeb1b3ece982f P 67856bddf7404c0c99446a985257bc7e: TxnStatusTablet state changed. Reason: New leader 86dae52b03fd4ebab95bbdb5d4a45d8a. Latest consensus state: current_term: 1 leader_uuid: "86dae52b03fd4ebab95bbdb5d4a45d8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8405f63549b4e06a48843842f6e1d3c" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 43109 } } peers { permanent_uuid: "86dae52b03fd4ebab95bbdb5d4a45d8a" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34809 } } peers { permanent_uuid: "67856bddf7404c0c99446a985257bc7e" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36029 } } }
I20260929 10:33:09.247750 17317 tablet_replica.cc:442] T 4867362bfccc40f6867eeb1b3ece982f P 67856bddf7404c0c99446a985257bc7e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 10:33:09.249313 17317 tablet_replica.cc:439] T 4867362bfccc40f6867eeb1b3ece982f P 67856bddf7404c0c99446a985257bc7e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "86dae52b03fd4ebab95bbdb5d4a45d8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8405f63549b4e06a48843842f6e1d3c" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 43109 } } peers { permanent_uuid: "86dae52b03fd4ebab95bbdb5d4a45d8a" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34809 } } peers { permanent_uuid: "67856bddf7404c0c99446a985257bc7e" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36029 } } }
I20260929 10:33:09.249428 17317 tablet_replica.cc:442] T 4867362bfccc40f6867eeb1b3ece982f P 67856bddf7404c0c99446a985257bc7e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 10:33:09.250020 17355 tablet_replica.cc:439] T 4867362bfccc40f6867eeb1b3ece982f P 86dae52b03fd4ebab95bbdb5d4a45d8a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "86dae52b03fd4ebab95bbdb5d4a45d8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8405f63549b4e06a48843842f6e1d3c" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 43109 } } peers { permanent_uuid: "86dae52b03fd4ebab95bbdb5d4a45d8a" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34809 } } peers { permanent_uuid: "67856bddf7404c0c99446a985257bc7e" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36029 } } }
I20260929 10:33:09.250121 17355 tablet_replica.cc:442] T 4867362bfccc40f6867eeb1b3ece982f P 86dae52b03fd4ebab95bbdb5d4a45d8a: This TxnStatusTablet replica's current role is: LEADER
I20260929 10:33:09.250285 17355 tablet_replica.cc:439] T 4867362bfccc40f6867eeb1b3ece982f P 86dae52b03fd4ebab95bbdb5d4a45d8a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "86dae52b03fd4ebab95bbdb5d4a45d8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8405f63549b4e06a48843842f6e1d3c" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 43109 } } peers { permanent_uuid: "86dae52b03fd4ebab95bbdb5d4a45d8a" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34809 } } peers { permanent_uuid: "67856bddf7404c0c99446a985257bc7e" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36029 } } }
I20260929 10:33:09.250379 17355 tablet_replica.cc:442] T 4867362bfccc40f6867eeb1b3ece982f P 86dae52b03fd4ebab95bbdb5d4a45d8a: This TxnStatusTablet replica's current role is: LEADER
I20260929 10:33:09.251276 17325 mvcc.cc:205] Tried to move back new op lower bound from 7334617043939348480 to 7334617043835731968. Current Snapshot: MvccSnapshot[applied={T|T < 7334617043939348480}]
I20260929 10:33:09.251437 17322 tablet_replica.cc:439] T 4867362bfccc40f6867eeb1b3ece982f P c8405f63549b4e06a48843842f6e1d3c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "86dae52b03fd4ebab95bbdb5d4a45d8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8405f63549b4e06a48843842f6e1d3c" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 43109 } } peers { permanent_uuid: "86dae52b03fd4ebab95bbdb5d4a45d8a" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34809 } } peers { permanent_uuid: "67856bddf7404c0c99446a985257bc7e" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36029 } } }
I20260929 10:33:09.251543 17322 tablet_replica.cc:442] T 4867362bfccc40f6867eeb1b3ece982f P c8405f63549b4e06a48843842f6e1d3c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 10:33:09.259704 17307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:09.255420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56090 (local address 127.15.82.67:36029)
0929 10:33:09.255558 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:09.255562 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:09.255626 (+ 64us) server_negotiation.cc:407] Connection header received
0929 10:33:09.255799 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:09.255802 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:09.255852 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:09.255961 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:33:09.256230 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:09.256724 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:09.257860 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:09.258067 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:09.258135 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:09.258480 (+ 345us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:33:09.258900 (+ 420us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:33:09.259026 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:09.259178 (+ 152us) server_negotiation.cc:299] Negotiation successful
0929 10:33:09.259236 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260929 10:33:09.271595 17365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:09.267806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.66:34809 (local address 127.15.82.67:43699)
0929 10:33:09.268138 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:09.268156 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:09.268259 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:09.268510 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:09.268513 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:09.268519 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 10:33:09.268626 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:09.268631 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:09.269625 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:09.269628 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:09.270265 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:09.270283 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:09.270892 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:09.270895 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:09.270972 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:09.271014 (+ 42us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 10:33:09.271335 (+ 321us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 10:33:09.271341 (+ 6us) client_negotiation.cc:769] Sending connection context
0929 10:33:09.271403 (+ 62us) client_negotiation.cc:240] Negotiation successful
0929 10:33:09.271463 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":126,"threads_started":1}
I20260929 10:33:09.271605 17306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:09.267909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.67:43699 (local address 127.15.82.66:34809)
0929 10:33:09.268035 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:09.268040 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:09.268237 (+ 197us) server_negotiation.cc:407] Connection header received
0929 10:33:09.268338 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:09.268341 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:09.268393 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:09.268479 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:33:09.268727 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:09.269503 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:09.270424 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:09.270767 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:09.270879 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:09.271152 (+ 273us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:33:09.271229 (+ 77us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:33:09.271301 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:09.271430 (+ 129us) server_negotiation.cc:299] Negotiation successful
0929 10:33:09.271481 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260929 10:33:09.278882 16962 raft_consensus.cc:1261] T b964812fc7394af9803f448c79366a68 P c8405f63549b4e06a48843842f6e1d3c [term 1 FOLLOWER]: Refusing update from remote peer 67856bddf7404c0c99446a985257bc7e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:33:09.278961 17101 raft_consensus.cc:1261] T b964812fc7394af9803f448c79366a68 P 86dae52b03fd4ebab95bbdb5d4a45d8a [term 1 FOLLOWER]: Refusing update from remote peer 67856bddf7404c0c99446a985257bc7e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:33:09.279312 17317 consensus_queue.cc:1035] T b964812fc7394af9803f448c79366a68 P 67856bddf7404c0c99446a985257bc7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8405f63549b4e06a48843842f6e1d3c" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 43109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:33:09.279698 17317 consensus_queue.cc:1035] T b964812fc7394af9803f448c79366a68 P 67856bddf7404c0c99446a985257bc7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "86dae52b03fd4ebab95bbdb5d4a45d8a" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:33:09.301260 17370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:09.297432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.67:36029 (local address 127.15.82.66:53973)
0929 10:33:09.297724 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:09.297736 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:09.297852 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:09.298113 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:09.298116 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:09.298137 (+ 21us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 10:33:09.298259 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:09.298265 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:09.299085 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:09.299091 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:09.300057 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:09.300078 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:09.300532 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:09.300535 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:09.300630 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:09.300680 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 10:33:09.300961 (+ 281us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 10:33:09.300969 (+ 8us) client_negotiation.cc:769] Sending connection context
0929 10:33:09.301050 (+ 81us) client_negotiation.cc:240] Negotiation successful
0929 10:33:09.301121 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":92,"threads_started":1}
I20260929 10:33:09.301237 17307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:09.297487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:53973 (local address 127.15.82.67:36029)
0929 10:33:09.297608 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:09.297613 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:09.297803 (+ 190us) server_negotiation.cc:407] Connection header received
0929 10:33:09.297966 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:09.297969 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:09.298020 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:09.298090 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:33:09.298369 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:09.298980 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:09.300183 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:09.300409 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:09.300532 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:09.300805 (+ 273us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:33:09.300880 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:33:09.300958 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:09.301058 (+ 100us) server_negotiation.cc:299] Negotiation successful
0929 10:33:09.301103 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260929 10:33:09.323067 16805 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46398:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:33:09.323241 16805 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46398:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:33:09.325181 16805 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c8c08b4557844cbbac9edc7f712c305f: Sending DeleteTablet for 3 replicas of tablet b964812fc7394af9803f448c79366a68
I20260929 10:33:09.325681 16942 tablet_service.cc:1514] Processing DeleteTablet for tablet b964812fc7394af9803f448c79366a68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:33:09 UTC) from {username='kudu', principal='kudu/127.15.82.126@KRBTEST.COM'} at 127.0.0.1:45714
I20260929 10:33:09.325865 17336 tablet_replica.cc:330] T b964812fc7394af9803f448c79366a68 P c8405f63549b4e06a48843842f6e1d3c: stopping tablet replica
I20260929 10:33:09.325874 17222 tablet_service.cc:1514] Processing DeleteTablet for tablet b964812fc7394af9803f448c79366a68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:33:09 UTC) from {username='kudu', principal='kudu/127.15.82.126@KRBTEST.COM'} at 127.0.0.1:56086
I20260929 10:33:09.326004 17336 raft_consensus.cc:2229] T b964812fc7394af9803f448c79366a68 P c8405f63549b4e06a48843842f6e1d3c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:33:09.326053 17335 tablet_replica.cc:330] T b964812fc7394af9803f448c79366a68 P 67856bddf7404c0c99446a985257bc7e: stopping tablet replica
I20260929 10:33:09.326198 17335 raft_consensus.cc:2229] T b964812fc7394af9803f448c79366a68 P 67856bddf7404c0c99446a985257bc7e [term 1 LEADER]: Raft consensus shutting down.
I20260929 10:33:09.326447 17082 tablet_service.cc:1514] Processing DeleteTablet for tablet b964812fc7394af9803f448c79366a68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:33:09 UTC) from {username='kudu', principal='kudu/127.15.82.126@KRBTEST.COM'} at 127.0.0.1:37520
I20260929 10:33:09.326571 17335 raft_consensus.cc:2258] T b964812fc7394af9803f448c79366a68 P 67856bddf7404c0c99446a985257bc7e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:33:09.326663 17336 raft_consensus.cc:2258] T b964812fc7394af9803f448c79366a68 P c8405f63549b4e06a48843842f6e1d3c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:33:09.326769 17334 tablet_replica.cc:330] T b964812fc7394af9803f448c79366a68 P 86dae52b03fd4ebab95bbdb5d4a45d8a: stopping tablet replica
I20260929 10:33:09.326881 17334 raft_consensus.cc:2229] T b964812fc7394af9803f448c79366a68 P 86dae52b03fd4ebab95bbdb5d4a45d8a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:33:09.326998 17334 raft_consensus.cc:2258] T b964812fc7394af9803f448c79366a68 P 86dae52b03fd4ebab95bbdb5d4a45d8a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:33:09.327751 17334 ts_tablet_manager.cc:1905] T b964812fc7394af9803f448c79366a68 P 86dae52b03fd4ebab95bbdb5d4a45d8a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:33:09.328433 17336 ts_tablet_manager.cc:1905] T b964812fc7394af9803f448c79366a68 P c8405f63549b4e06a48843842f6e1d3c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:33:09.330813 17334 ts_tablet_manager.cc:1918] T b964812fc7394af9803f448c79366a68 P 86dae52b03fd4ebab95bbdb5d4a45d8a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 10:33:09.330894 17334 log.cc:1196] T b964812fc7394af9803f448c79366a68 P 86dae52b03fd4ebab95bbdb5d4a45d8a: Deleting WAL directory at /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/minicluster-data/ts-1/wal/wals/b964812fc7394af9803f448c79366a68
I20260929 10:33:09.331158 17334 ts_tablet_manager.cc:1939] T b964812fc7394af9803f448c79366a68 P 86dae52b03fd4ebab95bbdb5d4a45d8a: Deleting consensus metadata
I20260929 10:33:09.331456 17336 ts_tablet_manager.cc:1918] T b964812fc7394af9803f448c79366a68 P c8405f63549b4e06a48843842f6e1d3c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 10:33:09.331537 17336 log.cc:1196] T b964812fc7394af9803f448c79366a68 P c8405f63549b4e06a48843842f6e1d3c: Deleting WAL directory at /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/minicluster-data/ts-0/wal/wals/b964812fc7394af9803f448c79366a68
I20260929 10:33:09.331787 17336 ts_tablet_manager.cc:1939] T b964812fc7394af9803f448c79366a68 P c8405f63549b4e06a48843842f6e1d3c: Deleting consensus metadata
I20260929 10:33:09.331907 16791 catalog_manager.cc:4841] TS 86dae52b03fd4ebab95bbdb5d4a45d8a (127.15.82.66:34809): tablet b964812fc7394af9803f448c79366a68 (table test-table [id=9f1739a065254426932a9502dae8498d]) successfully deleted
Sep 29 10:33:09 dist-test-slave-vg89 krb5kdc[16760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790677988, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.82.65@KRBTEST.COM
I20260929 10:33:09.339387 17335 ts_tablet_manager.cc:1905] T b964812fc7394af9803f448c79366a68 P 67856bddf7404c0c99446a985257bc7e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:33:09.341998 17335 ts_tablet_manager.cc:1918] T b964812fc7394af9803f448c79366a68 P 67856bddf7404c0c99446a985257bc7e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 10:33:09.342063 17335 log.cc:1196] T b964812fc7394af9803f448c79366a68 P 67856bddf7404c0c99446a985257bc7e: Deleting WAL directory at /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790677981592896-15689-0/minicluster-data/ts-2/wal/wals/b964812fc7394af9803f448c79366a68
I20260929 10:33:09.342293 17335 ts_tablet_manager.cc:1939] T b964812fc7394af9803f448c79366a68 P 67856bddf7404c0c99446a985257bc7e: Deleting consensus metadata
I20260929 10:33:09.343034 16792 catalog_manager.cc:4841] TS 67856bddf7404c0c99446a985257bc7e (127.15.82.67:36029): tablet b964812fc7394af9803f448c79366a68 (table test-table [id=9f1739a065254426932a9502dae8498d]) successfully deleted
I20260929 10:33:09.345440 16789 catalog_manager.cc:4841] TS c8405f63549b4e06a48843842f6e1d3c (127.15.82.65:43109): tablet b964812fc7394af9803f448c79366a68 (table test-table [id=9f1739a065254426932a9502dae8498d]) successfully deleted
I20260929 10:33:09.347395 17310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:09.328138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45744 (local address 127.15.82.65:43109)
0929 10:33:09.328246 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:09.328250 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:09.328879 (+ 629us) server_negotiation.cc:407] Connection header received
0929 10:33:09.329079 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:09.329082 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:09.329129 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:09.329179 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:09.330204 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:09.330976 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:09.333307 (+ 2331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:09.333629 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:09.333702 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:09.337793 (+ 4091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:09.337804 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:09.337819 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:09.337844 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:09.344653 (+ 6809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:09.345821 (+ 1168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:09.345828 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:09.345830 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:09.345930 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:09.346220 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:09.346222 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:09.346224 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:09.346538 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:09.346711 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:09.347095 (+ 384us) server_negotiation.cc:299] Negotiation successful
0929 10:33:09.347243 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
W20260929 10:33:09.348537 17002 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45744
I20260929 10:33:09.363423 17349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:09.350490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46422 (local address 127.15.82.126:40081)
0929 10:33:09.350617 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:09.350621 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:09.354665 (+ 4044us) server_negotiation.cc:407] Connection header received
0929 10:33:09.354837 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:09.354840 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:09.354887 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:09.354934 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:09.355802 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:09.356649 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:09.357760 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:09.357998 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:09.358114 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:09.359908 (+ 1794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:09.359912 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:09.359914 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:09.359950 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:09.361576 (+ 1626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:09.362142 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:09.362146 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:09.362148 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:09.362260 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:09.362567 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:09.362570 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:09.362572 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:09.362756 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:09.362897 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:09.363110 (+ 213us) server_negotiation.cc:299] Negotiation successful
0929 10:33:09.363242 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
W20260929 10:33:09.363855 16805 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46422
I20260929 10:33:09.365559 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 16877
I20260929 10:33:09.375005 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 17017
I20260929 10:33:09.382042 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 17157
I20260929 10:33:09.387915 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 16773
2026-09-29T10:33:09Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2900 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:33:09 dist-test-slave-vg89 krb5kdc[17379](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:33:09 dist-test-slave-vg89 krb5kdc[17379](info): set up 2 sockets
Sep 29 10:33:09 dist-test-slave-vg89 krb5kdc[17379](info): commencing operation
krb5kdc: starting...
W20260929 10:33:10.422693 15689 mini_kdc.cc:121] Time spent starting KDC: real 1.011s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Sep 29 10:33:10 dist-test-slave-vg89 krb5kdc[17379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790677990, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:33:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:33:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.82.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab.
Entry for principal kudu/127.15.82.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab.
I20260929 10:33:10.545212 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:39349
--webserver_interface=127.15.82.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab
--principal=kudu/127.15.82.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:44415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.82.126:39349
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:33:10.656324 17392 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:33:10.656597 17392 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:33:10.656637 17392 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:33:10.660701 17392 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:33:10.660754 17392 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260929 10:33:10.660771 17392 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:33:10.660790 17392 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:33:10.660837 17392 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:33:10.660874 17392 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:33:10.666172 17392 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:44415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.82.126:39349
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab
--principal=kudu/127.15.82.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:39349
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.82.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17392
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:33:10.667279 17392 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:33:10.668313 17392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:33:10.676936 17400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:10.676949 17398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:10.676985 17392 server_base.cc:1034] running on GCE node
W20260929 10:33:10.676921 17397 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:10.677770 17392 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:33:10.678884 17392 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:33:10.680094 17392 hybrid_clock.cc:648] HybridClock initialized: now 1790677990680055 us; error 52 us; skew 500 ppm
Sep 29 10:33:10 dist-test-slave-vg89 krb5kdc[17379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790677990, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:33:10.682866 17392 init.cc:377] Logged in from keytab as kudu/127.15.82.126@KRBTEST.COM (short username kudu)
I20260929 10:33:10.683691 17392 webserver.cc:460] Webserver started at http://127.15.82.126:35723/ using document root <none> and password file <none>
I20260929 10:33:10.684201 17392 fs_manager.cc:359] Metadata directory not provided
I20260929 10:33:10.684257 17392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:33:10.684418 17392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:33:10.686455 17392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/master-0/data/instance:
uuid: "3639d14359334e7fbfb18a16004331c8"
format_stamp: "Formatted at 2026-09-29 10:33:10 on dist-test-slave-vg89"
server_key: "522f92fb1d8e57741865d5a53cfc272f"
server_key_iv: "e5ae6cf8667b148dd9aecae7fb1b8783"
server_key_version: "encryptionkey@0"
I20260929 10:33:10.687004 17392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/master-0/wal/instance:
uuid: "3639d14359334e7fbfb18a16004331c8"
format_stamp: "Formatted at 2026-09-29 10:33:10 on dist-test-slave-vg89"
server_key: "522f92fb1d8e57741865d5a53cfc272f"
server_key_iv: "e5ae6cf8667b148dd9aecae7fb1b8783"
server_key_version: "encryptionkey@0"
I20260929 10:33:10.690730 17392 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260929 10:33:10.693128 17407 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:10.694331 17392 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 10:33:10.694522 17392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/master-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/master-0/wal
uuid: "3639d14359334e7fbfb18a16004331c8"
format_stamp: "Formatted at 2026-09-29 10:33:10 on dist-test-slave-vg89"
server_key: "522f92fb1d8e57741865d5a53cfc272f"
server_key_iv: "e5ae6cf8667b148dd9aecae7fb1b8783"
server_key_version: "encryptionkey@0"
I20260929 10:33:10.694643 17392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:33:10.715144 17392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:33:10.718366 17392 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:33:10.718619 17392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:33:10.726531 17392 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.126:39349
I20260929 10:33:10.726564 17469 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.126:39349 every 8 connection(s)
I20260929 10:33:10.727488 17392 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
I20260929 10:33:10.730415 17470 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:33:10.731858 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 17392
I20260929 10:33:10.731952 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/master-0/wal/instance
I20260929 10:33:10.732247 15689 external_mini_cluster.cc:1428] Setting key 7805b8d137a47d5e324fff8f16d60d05
I20260929 10:33:10.738449 17470 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3639d14359334e7fbfb18a16004331c8: Bootstrap starting.
I20260929 10:33:10.741505 17470 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3639d14359334e7fbfb18a16004331c8: Neither blocks nor log segments found. Creating new log.
I20260929 10:33:10.742355 17470 log.cc:824] T 00000000000000000000000000000000 P 3639d14359334e7fbfb18a16004331c8: Log is configured to *not* fsync() on all Append() calls
Sep 29 10:33:10 dist-test-slave-vg89 krb5kdc[17379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790677990, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:33:10.744835 17470 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3639d14359334e7fbfb18a16004331c8: No bootstrap required, opened a new log
I20260929 10:33:10.748502 17473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:10.733761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51566 (local address 127.15.82.126:39349)
0929 10:33:10.734223 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:10.734248 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:10.734300 (+ 52us) server_negotiation.cc:407] Connection header received
0929 10:33:10.735320 (+ 1020us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:10.735346 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:10.735816 (+ 470us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:10.736237 (+ 421us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:10.736913 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:10.738611 (+ 1698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:10.739533 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:10.740354 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:10.740541 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:10.743273 (+ 2732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:10.743280 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:10.743303 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:10.743330 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:10.745591 (+ 2261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:10.746153 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:10.746157 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:10.746162 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:10.746271 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:10.746618 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:10.746620 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:10.746621 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:10.746994 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:10.747137 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:10.747476 (+ 339us) server_negotiation.cc:299] Negotiation successful
0929 10:33:10.747768 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":116,"threads_started":1}
I20260929 10:33:10.748942 17470 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3639d14359334e7fbfb18a16004331c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3639d14359334e7fbfb18a16004331c8" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 39349 } }
I20260929 10:33:10.749210 17470 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3639d14359334e7fbfb18a16004331c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:33:10.749311 17470 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3639d14359334e7fbfb18a16004331c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3639d14359334e7fbfb18a16004331c8, State: Initialized, Role: FOLLOWER
I20260929 10:33:10.749840 17470 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3639d14359334e7fbfb18a16004331c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3639d14359334e7fbfb18a16004331c8" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 39349 } }
I20260929 10:33:10.750036 17470 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3639d14359334e7fbfb18a16004331c8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:33:10.750119 17470 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3639d14359334e7fbfb18a16004331c8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:33:10.750228 17470 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3639d14359334e7fbfb18a16004331c8 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:33:10.751489 17470 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3639d14359334e7fbfb18a16004331c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3639d14359334e7fbfb18a16004331c8" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 39349 } }
I20260929 10:33:10.751803 17470 leader_election.cc:302] T 00000000000000000000000000000000 P 3639d14359334e7fbfb18a16004331c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3639d14359334e7fbfb18a16004331c8; no voters:
I20260929 10:33:10.752076 17470 leader_election.cc:288] T 00000000000000000000000000000000 P 3639d14359334e7fbfb18a16004331c8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:33:10.752168 17475 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3639d14359334e7fbfb18a16004331c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:33:10.752424 17475 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3639d14359334e7fbfb18a16004331c8 [term 1 LEADER]: Becoming Leader. State: Replica: 3639d14359334e7fbfb18a16004331c8, State: Running, Role: LEADER
I20260929 10:33:10.752728 17475 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3639d14359334e7fbfb18a16004331c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3639d14359334e7fbfb18a16004331c8" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 39349 } }
I20260929 10:33:10.753499 17470 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3639d14359334e7fbfb18a16004331c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:33:10.754526 17477 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3639d14359334e7fbfb18a16004331c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3639d14359334e7fbfb18a16004331c8. Latest consensus state: current_term: 1 leader_uuid: "3639d14359334e7fbfb18a16004331c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3639d14359334e7fbfb18a16004331c8" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 39349 } } }
I20260929 10:33:10.754503 17476 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3639d14359334e7fbfb18a16004331c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3639d14359334e7fbfb18a16004331c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3639d14359334e7fbfb18a16004331c8" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 39349 } } }
I20260929 10:33:10.754639 17476 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3639d14359334e7fbfb18a16004331c8 [sys.catalog]: This master's current role is: LEADER
I20260929 10:33:10.754639 17477 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3639d14359334e7fbfb18a16004331c8 [sys.catalog]: This master's current role is: LEADER
I20260929 10:33:10.755010 17484 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:33:10.758118 17484 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:33:10.763795 17484 catalog_manager.cc:1347] Generated new cluster ID: ffba5c58d57743149e75e2ec7ae9d87d
I20260929 10:33:10.763870 17484 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:33:10.775341 17484 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:33:10.776237 17484 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:33:10.784411 17484 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3639d14359334e7fbfb18a16004331c8: Generated new TSK 0
I20260929 10:33:10.785058 17484 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.82.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab.
Entry for principal kudu/127.15.82.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab.
I20260929 10:33:10.817646 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.65:0
--local_ip_for_outbound_sockets=127.15.82.65
--webserver_interface=127.15.82.65
--webserver_port=0
--tserver_master_addrs=127.15.82.126:39349
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab
--principal=kudu/127.15.82.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:44415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:33:10.929445 17496 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:33:10.929726 17496 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:33:10.929775 17496 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:33:10.934016 17496 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:33:10.934082 17496 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:33:10.934123 17496 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260929 10:33:10.934137 17496 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 10:33:10.934218 17496 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.65
I20260929 10:33:10.939636 17496 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:44415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab
--principal=kudu/127.15.82.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.82.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.82.126:39349
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17496
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.65
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:33:10.940811 17496 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:33:10.941871 17496 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:33:10.951586 17502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:10.951586 17504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:10.951627 17501 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:10.951921 17496 server_base.cc:1034] running on GCE node
I20260929 10:33:10.952332 17496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:33:10.953003 17496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:33:10.954159 17496 hybrid_clock.cc:648] HybridClock initialized: now 1790677990954142 us; error 28 us; skew 500 ppm
Sep 29 10:33:10 dist-test-slave-vg89 krb5kdc[17379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790677990, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:33:10.957108 17496 init.cc:377] Logged in from keytab as kudu/127.15.82.65@KRBTEST.COM (short username kudu)
I20260929 10:33:10.957970 17496 webserver.cc:460] Webserver started at http://127.15.82.65:43459/ using document root <none> and password file <none>
I20260929 10:33:10.958600 17496 fs_manager.cc:359] Metadata directory not provided
I20260929 10:33:10.958688 17496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:33:10.958918 17496 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:33:10.960927 17496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/ts-0/data/instance:
uuid: "1f1282b07b1f4d5c95e009991f21abbd"
format_stamp: "Formatted at 2026-09-29 10:33:10 on dist-test-slave-vg89"
server_key: "03c7b0e8f614a0809b46d22e5eeaaf74"
server_key_iv: "5c8672a3e36b455bdcd4398572422492"
server_key_version: "encryptionkey@0"
I20260929 10:33:10.961488 17496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/ts-0/wal/instance:
uuid: "1f1282b07b1f4d5c95e009991f21abbd"
format_stamp: "Formatted at 2026-09-29 10:33:10 on dist-test-slave-vg89"
server_key: "03c7b0e8f614a0809b46d22e5eeaaf74"
server_key_iv: "5c8672a3e36b455bdcd4398572422492"
server_key_version: "encryptionkey@0"
I20260929 10:33:10.965222 17496 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.006s
I20260929 10:33:10.967624 17511 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:10.968775 17496 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260929 10:33:10.968907 17496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/ts-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/ts-0/wal
uuid: "1f1282b07b1f4d5c95e009991f21abbd"
format_stamp: "Formatted at 2026-09-29 10:33:10 on dist-test-slave-vg89"
server_key: "03c7b0e8f614a0809b46d22e5eeaaf74"
server_key_iv: "5c8672a3e36b455bdcd4398572422492"
server_key_version: "encryptionkey@0"
I20260929 10:33:10.969022 17496 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:33:10.994243 17496 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:33:10.997787 17496 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:33:10.997987 17496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:33:10.999524 17496 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:33:10.999596 17496 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:10.999670 17496 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:33:10.999713 17496 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 10:33:11 dist-test-slave-vg89 krb5kdc[17379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790677990, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.65@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:33:11.010738 17496 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.65:45387
I20260929 10:33:11.010814 17626 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.65:45387 every 8 connection(s)
I20260929 10:33:11.011873 17496 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
I20260929 10:33:11.013995 17473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:11.000406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.65:34359 (local address 127.15.82.126:39349)
0929 10:33:11.000539 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:11.000543 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:11.001609 (+ 1066us) server_negotiation.cc:407] Connection header received
0929 10:33:11.002772 (+ 1163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:11.002775 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:11.002864 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:11.002942 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:11.004663 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:11.005595 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:11.007074 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:11.007516 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:11.007632 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:11.010206 (+ 2574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:11.010212 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:11.010215 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:11.010242 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:11.012036 (+ 1794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:11.012598 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:11.012601 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:11.012603 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:11.012700 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:11.013068 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:11.013070 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:11.013071 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:11.013246 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:11.013321 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:11.013750 (+ 429us) server_negotiation.cc:299] Negotiation successful
0929 10:33:11.013863 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260929 10:33:11.014293 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 17496
I20260929 10:33:11.014434 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/ts-0/wal/instance
I20260929 10:33:11.014747 15689 external_mini_cluster.cc:1428] Setting key 29ed9ac2dc3e8aaab16cf80474c0855e
I20260929 10:33:11.015020 17520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:11.000978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:39349 (local address 127.15.82.65:34359)
0929 10:33:11.001455 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:11.001508 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:11.002501 (+ 993us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:11.003389 (+ 888us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:11.003400 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:11.003800 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:11.004463 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:11.004492 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:11.005757 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:11.005762 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:11.006947 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:11.006957 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:11.007659 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:11.007661 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:11.007797 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:11.008358 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:11.008374 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:11.010022 (+ 1648us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:11.012142 (+ 2120us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:11.012147 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:11.012158 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:11.012471 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:11.012803 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:11.012807 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:11.012808 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:11.012984 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:11.013332 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:11.013337 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:11.013552 (+ 215us) client_negotiation.cc:769] Sending connection context
0929 10:33:11.013702 (+ 150us) client_negotiation.cc:240] Negotiation successful
0929 10:33:11.013936 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":124,"threads_started":1}
I20260929 10:33:11.016366 17627 heartbeater.cc:347] Connected to a master server at 127.15.82.126:39349
I20260929 10:33:11.016501 17627 heartbeater.cc:464] Registering TS with master...
I20260929 10:33:11.016964 17627 heartbeater.cc:511] Master 127.15.82.126:39349 requested a full tablet report, sending...
I20260929 10:33:11.018663 17424 ts_manager.cc:194] Registered new tserver with Master: 1f1282b07b1f4d5c95e009991f21abbd (127.15.82.65:45387)
I20260929 10:33:11.020295 17424 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.82.65@KRBTEST.COM'} at 127.15.82.65:34359
WARNING: no policy specified for kudu/127.15.82.66@KRBTEST.COM; defaulting to no policy
I20260929 10:33:11.026175 17634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:11.018719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:39349 (local address 127.15.82.65:56939)
0929 10:33:11.019011 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:11.019023 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:11.019132 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:11.019357 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:11.019359 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:11.019564 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:11.019657 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:11.019662 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:11.020429 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:11.020433 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:11.021073 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:11.021082 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:11.021749 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:11.021753 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:11.021840 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:11.022458 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:11.022472 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:11.022899 (+ 427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:11.024787 (+ 1888us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:11.024790 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:11.024791 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:11.024993 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:11.025295 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:11.025297 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:11.025299 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:11.025370 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:11.025766 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:11.025768 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:11.025838 (+ 70us) client_negotiation.cc:769] Sending connection context
0929 10:33:11.025911 (+ 73us) client_negotiation.cc:240] Negotiation successful
0929 10:33:11.026024 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":80,"threads_started":1}
I20260929 10:33:11.026273 17473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:11.018759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.65:56939 (local address 127.15.82.126:39349)
0929 10:33:11.018871 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:11.018876 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:11.019094 (+ 218us) server_negotiation.cc:407] Connection header received
0929 10:33:11.019207 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:11.019210 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:11.019251 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:11.019342 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:11.019771 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:11.020298 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:11.021222 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:11.021614 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:11.021735 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:11.023043 (+ 1308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:11.023049 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:11.023054 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:11.023086 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:11.024685 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:11.025092 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:11.025098 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:11.025101 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:11.025192 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:11.025460 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:11.025463 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:11.025465 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:11.025671 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:11.025772 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:11.025977 (+ 205us) server_negotiation.cc:299] Negotiation successful
0929 10:33:11.026107 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab.
Entry for principal kudu/127.15.82.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab.
I20260929 10:33:11.043490 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.66:0
--local_ip_for_outbound_sockets=127.15.82.66
--webserver_interface=127.15.82.66
--webserver_port=0
--tserver_master_addrs=127.15.82.126:39349
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab
--principal=kudu/127.15.82.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:44415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:33:11.153213 17636 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:33:11.153486 17636 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:33:11.153533 17636 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:33:11.157904 17636 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:33:11.157963 17636 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:33:11.158005 17636 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260929 10:33:11.158020 17636 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 10:33:11.158056 17636 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.66
I20260929 10:33:11.163362 17636 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:44415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab
--principal=kudu/127.15.82.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.82.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.82.126:39349
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17636
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.66
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:33:11.164367 17636 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:33:11.165350 17636 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:33:11.175109 17644 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:11.175125 17641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:11.175195 17636 server_base.cc:1034] running on GCE node
W20260929 10:33:11.175122 17642 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:11.175742 17636 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:33:11.176384 17636 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:33:11.177551 17636 hybrid_clock.cc:648] HybridClock initialized: now 1790677991177534 us; error 28 us; skew 500 ppm
Sep 29 10:33:11 dist-test-slave-vg89 krb5kdc[17379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790677991, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:33:11.180343 17636 init.cc:377] Logged in from keytab as kudu/127.15.82.66@KRBTEST.COM (short username kudu)
I20260929 10:33:11.181181 17636 webserver.cc:460] Webserver started at http://127.15.82.66:45075/ using document root <none> and password file <none>
I20260929 10:33:11.181727 17636 fs_manager.cc:359] Metadata directory not provided
I20260929 10:33:11.181785 17636 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:33:11.182027 17636 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:33:11.184047 17636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/ts-1/data/instance:
uuid: "7f912be145e249078c19a61fb9f35c53"
format_stamp: "Formatted at 2026-09-29 10:33:11 on dist-test-slave-vg89"
server_key: "b84e73aadafc0c3a9186d88cb2fa387c"
server_key_iv: "29c96d73e605c8e03185738b3d2b0080"
server_key_version: "encryptionkey@0"
I20260929 10:33:11.184582 17636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/ts-1/wal/instance:
uuid: "7f912be145e249078c19a61fb9f35c53"
format_stamp: "Formatted at 2026-09-29 10:33:11 on dist-test-slave-vg89"
server_key: "b84e73aadafc0c3a9186d88cb2fa387c"
server_key_iv: "29c96d73e605c8e03185738b3d2b0080"
server_key_version: "encryptionkey@0"
I20260929 10:33:11.188194 17636 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260929 10:33:11.190662 17651 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:11.191855 17636 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 10:33:11.191983 17636 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/ts-1/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/ts-1/wal
uuid: "7f912be145e249078c19a61fb9f35c53"
format_stamp: "Formatted at 2026-09-29 10:33:11 on dist-test-slave-vg89"
server_key: "b84e73aadafc0c3a9186d88cb2fa387c"
server_key_iv: "29c96d73e605c8e03185738b3d2b0080"
server_key_version: "encryptionkey@0"
I20260929 10:33:11.192085 17636 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:33:11.221972 17636 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:33:11.224925 17636 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:33:11.225142 17636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:33:11.226620 17636 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:33:11.226670 17636 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:11.226727 17636 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:33:11.226758 17636 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 10:33:11 dist-test-slave-vg89 krb5kdc[17379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790677991, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.66@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:33:11.237622 17636 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.66:43273
I20260929 10:33:11.237699 17766 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.66:43273 every 8 connection(s)
I20260929 10:33:11.238695 17636 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
I20260929 10:33:11.239740 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 17636
I20260929 10:33:11.239861 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/ts-1/wal/instance
I20260929 10:33:11.240166 15689 external_mini_cluster.cc:1428] Setting key 92645980f0d62610bbacf2a698d01256
I20260929 10:33:11.241151 17473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:11.227498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:57573 (local address 127.15.82.126:39349)
0929 10:33:11.227648 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:11.227653 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:11.228523 (+ 870us) server_negotiation.cc:407] Connection header received
0929 10:33:11.229692 (+ 1169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:11.229695 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:11.229754 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:11.229851 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:11.231520 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:11.232383 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:11.233602 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:11.233958 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:11.234111 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:11.237188 (+ 3077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:11.237192 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:11.237195 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:11.237223 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:11.238798 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:11.239406 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:11.239411 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:11.239413 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:11.239545 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:11.239948 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:11.239952 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:11.239953 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:11.240159 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:11.240288 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:11.240865 (+ 577us) server_negotiation.cc:299] Negotiation successful
0929 10:33:11.241004 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260929 10:33:11.242125 17660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:11.227904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:39349 (local address 127.15.82.66:57573)
0929 10:33:11.228352 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:11.228403 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:11.229401 (+ 998us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:11.230080 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:11.230089 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:11.230613 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:11.231347 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:11.231369 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:11.232516 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:11.232519 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:11.233455 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:11.233465 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:11.234170 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:11.234179 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:11.234429 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:11.234985 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:11.235009 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:11.236989 (+ 1980us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:11.238905 (+ 1916us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:11.238911 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:11.238925 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:11.239276 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:11.239647 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:11.239650 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:11.239653 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:11.239843 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:11.240302 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:11.240310 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:11.240602 (+ 292us) client_negotiation.cc:769] Sending connection context
0929 10:33:11.240819 (+ 217us) client_negotiation.cc:240] Negotiation successful
0929 10:33:11.241116 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":122,"threads_started":1}
I20260929 10:33:11.243525 17767 heartbeater.cc:347] Connected to a master server at 127.15.82.126:39349
I20260929 10:33:11.243661 17767 heartbeater.cc:464] Registering TS with master...
I20260929 10:33:11.244210 17767 heartbeater.cc:511] Master 127.15.82.126:39349 requested a full tablet report, sending...
I20260929 10:33:11.245280 17424 ts_manager.cc:194] Registered new tserver with Master: 7f912be145e249078c19a61fb9f35c53 (127.15.82.66:43273)
I20260929 10:33:11.246038 17424 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.82.66@KRBTEST.COM'} at 127.15.82.66:57573
WARNING: no policy specified for kudu/127.15.82.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.67@KRBTEST.COM" created.
I20260929 10:33:11.253974 17473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:11.245825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:47799 (local address 127.15.82.126:39349)
0929 10:33:11.245921 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:11.245925 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:11.246666 (+ 741us) server_negotiation.cc:407] Connection header received
0929 10:33:11.246774 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:11.246777 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:11.246829 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:11.246914 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:11.247405 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:11.248297 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:11.249084 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:11.249458 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:11.249585 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:11.250890 (+ 1305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:11.250894 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:11.250901 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:11.250927 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:11.252395 (+ 1468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:11.252767 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:11.252769 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:11.252770 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:11.252837 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:11.253127 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:11.253130 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:11.253132 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:11.253335 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:11.253462 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:11.253703 (+ 241us) server_negotiation.cc:299] Negotiation successful
0929 10:33:11.253810 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260929 10:33:11.253988 17774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:11.245782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:39349 (local address 127.15.82.66:47799)
0929 10:33:11.246571 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:11.246583 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:11.246694 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:11.246943 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:11.246946 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:11.247153 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:11.247273 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:11.247279 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:11.248426 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:11.248428 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:11.248969 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:11.248976 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:11.249569 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:11.249572 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:11.249637 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:11.250206 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:11.250226 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:11.250719 (+ 493us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:11.252490 (+ 1771us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:11.252492 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:11.252494 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:11.252683 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:11.252936 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:11.252940 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:11.252942 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:11.253044 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:11.253438 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:11.253441 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:11.253531 (+ 90us) client_negotiation.cc:769] Sending connection context
0929 10:33:11.253641 (+ 110us) client_negotiation.cc:240] Negotiation successful
0929 10:33:11.253786 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":707,"thread_start_us":82,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab.
Entry for principal kudu/127.15.82.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab.
I20260929 10:33:11.269767 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.67:0
--local_ip_for_outbound_sockets=127.15.82.67
--webserver_interface=127.15.82.67
--webserver_port=0
--tserver_master_addrs=127.15.82.126:39349
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab
--principal=kudu/127.15.82.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:44415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:33:11.381099 17776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:33:11.381373 17776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:33:11.381421 17776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:33:11.385560 17776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:33:11.385628 17776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:33:11.385707 17776 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260929 10:33:11.385741 17776 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 10:33:11.385830 17776 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.67
I20260929 10:33:11.391175 17776 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:44415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab
--principal=kudu/127.15.82.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.82.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.82.126:39349
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17776
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.67
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:33:11.392275 17776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:33:11.393591 17776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:33:11.403738 17781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:11.403836 17776 server_base.cc:1034] running on GCE node
W20260929 10:33:11.403733 17784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:11.403733 17782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:11.404424 17776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:33:11.405072 17776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:33:11.406232 17776 hybrid_clock.cc:648] HybridClock initialized: now 1790677991406214 us; error 28 us; skew 500 ppm
Sep 29 10:33:11 dist-test-slave-vg89 krb5kdc[17379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790677991, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:33:11.408982 17776 init.cc:377] Logged in from keytab as kudu/127.15.82.67@KRBTEST.COM (short username kudu)
I20260929 10:33:11.409934 17776 webserver.cc:460] Webserver started at http://127.15.82.67:36361/ using document root <none> and password file <none>
I20260929 10:33:11.410581 17776 fs_manager.cc:359] Metadata directory not provided
I20260929 10:33:11.410646 17776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:33:11.410890 17776 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:33:11.412882 17776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/ts-2/data/instance:
uuid: "2b9cd0624fa84bc5b1a0de85bce66a72"
format_stamp: "Formatted at 2026-09-29 10:33:11 on dist-test-slave-vg89"
server_key: "bb35890b92d135ef03d2b453f586664e"
server_key_iv: "bf0d3a38080508a6071c9e81a78ac38d"
server_key_version: "encryptionkey@0"
I20260929 10:33:11.413465 17776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/ts-2/wal/instance:
uuid: "2b9cd0624fa84bc5b1a0de85bce66a72"
format_stamp: "Formatted at 2026-09-29 10:33:11 on dist-test-slave-vg89"
server_key: "bb35890b92d135ef03d2b453f586664e"
server_key_iv: "bf0d3a38080508a6071c9e81a78ac38d"
server_key_version: "encryptionkey@0"
I20260929 10:33:11.417181 17776 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260929 10:33:11.419704 17791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:11.420971 17776 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260929 10:33:11.421106 17776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/ts-2/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/ts-2/wal
uuid: "2b9cd0624fa84bc5b1a0de85bce66a72"
format_stamp: "Formatted at 2026-09-29 10:33:11 on dist-test-slave-vg89"
server_key: "bb35890b92d135ef03d2b453f586664e"
server_key_iv: "bf0d3a38080508a6071c9e81a78ac38d"
server_key_version: "encryptionkey@0"
I20260929 10:33:11.421219 17776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:33:11.439834 17776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:33:11.442706 17776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:33:11.442935 17776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:33:11.444541 17776 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:33:11.444612 17776 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:11.444712 17776 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:33:11.444748 17776 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:11.455251 17776 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.67:40743
Sep 29 10:33:11 dist-test-slave-vg89 krb5kdc[17379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790677991, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.67@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:33:11.456378 17776 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
I20260929 10:33:11.456681 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 17776
I20260929 10:33:11.456805 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/ts-2/wal/instance
I20260929 10:33:11.457157 15689 external_mini_cluster.cc:1428] Setting key 911fa321b8fb1fc529f89e79dfac4c64
I20260929 10:33:11.456990 17906 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.67:40743 every 8 connection(s)
I20260929 10:33:11.460016 17473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:11.445229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.67:42635 (local address 127.15.82.126:39349)
0929 10:33:11.445391 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:11.445398 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:11.446734 (+ 1336us) server_negotiation.cc:407] Connection header received
0929 10:33:11.447832 (+ 1098us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:11.447835 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:11.447895 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:11.447994 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:11.449690 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:11.450573 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:11.452378 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:11.452604 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:11.452687 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:11.456050 (+ 3363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:11.456054 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:11.456056 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:11.456086 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:11.457728 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:11.458315 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:11.458320 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:11.458325 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:11.458464 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:11.458850 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:11.458853 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:11.458854 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:11.459035 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:11.459114 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:11.459779 (+ 665us) server_negotiation.cc:299] Negotiation successful
0929 10:33:11.459893 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260929 10:33:11.460922 17800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:11.446094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:39349 (local address 127.15.82.67:42635)
0929 10:33:11.446587 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:11.446638 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:11.447616 (+ 978us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:11.448204 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:11.448213 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:11.448732 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:11.449500 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:11.449520 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:11.450709 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:11.450716 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:11.452255 (+ 1539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:11.452269 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:11.452761 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:11.452765 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:11.452987 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:11.453771 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:11.453794 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:11.455787 (+ 1993us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:11.457845 (+ 2058us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:11.457851 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:11.457866 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:11.458205 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:11.458554 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:11.458557 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:11.458558 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:11.458745 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:11.459144 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:11.459151 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:11.459447 (+ 296us) client_negotiation.cc:769] Sending connection context
0929 10:33:11.459681 (+ 234us) client_negotiation.cc:240] Negotiation successful
0929 10:33:11.460079 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":133,"threads_started":1}
I20260929 10:33:11.462137 17907 heartbeater.cc:347] Connected to a master server at 127.15.82.126:39349
I20260929 10:33:11.462267 17907 heartbeater.cc:464] Registering TS with master...
I20260929 10:33:11.462816 17907 heartbeater.cc:511] Master 127.15.82.126:39349 requested a full tablet report, sending...
I20260929 10:33:11.463914 17423 ts_manager.cc:194] Registered new tserver with Master: 2b9cd0624fa84bc5b1a0de85bce66a72 (127.15.82.67:40743)
I20260929 10:33:11.464687 17423 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.82.67@KRBTEST.COM'} at 127.15.82.67:42635
I20260929 10:33:11.471199 15689 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 10:33:11.471521 17913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:11.464389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:39349 (local address 127.15.82.67:40357)
0929 10:33:11.464641 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:11.464656 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:11.464755 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:11.465138 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:11.465141 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:11.465339 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:11.465439 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:11.465444 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:11.466449 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:11.466453 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:11.467024 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:11.467032 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:11.467470 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:11.467473 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:11.467544 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:11.467967 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:11.467978 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:11.468270 (+ 292us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:11.469893 (+ 1623us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:11.469896 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:11.469898 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:11.470151 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:11.470457 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:11.470461 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:11.470464 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:11.470569 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:11.470980 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:11.470983 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:11.471073 (+ 90us) client_negotiation.cc:769] Sending connection context
0929 10:33:11.471215 (+ 142us) client_negotiation.cc:240] Negotiation successful
0929 10:33:11.471358 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":99,"threads_started":1}
I20260929 10:33:11.471526 17473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:11.464439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.67:40357 (local address 127.15.82.126:39349)
0929 10:33:11.464577 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:11.464584 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:11.464726 (+ 142us) server_negotiation.cc:407] Connection header received
0929 10:33:11.464883 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:11.464886 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:11.464924 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:11.465015 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:11.465558 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:11.466271 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:11.467168 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:11.467370 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:11.467454 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:11.468388 (+ 934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:11.468390 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:11.468392 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:11.468413 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:11.469802 (+ 1389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:11.470244 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:11.470247 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:11.470249 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:11.470329 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:11.470666 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:11.470668 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:11.470669 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:11.470878 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:11.470979 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:11.471281 (+ 302us) server_negotiation.cc:299] Negotiation successful
0929 10:33:11.471392 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
Sep 29 10:33:11 dist-test-slave-vg89 krb5kdc[17379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790677991, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 29 10:33:11 dist-test-slave-vg89 krb5kdc[17379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790677991, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:33:11.497556 17473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:11.489258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51582 (local address 127.15.82.126:39349)
0929 10:33:11.489391 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:11.489395 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:11.489407 (+ 12us) server_negotiation.cc:407] Connection header received
0929 10:33:11.489458 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:11.489460 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:11.489500 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:11.489570 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:11.490256 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:11.491077 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:11.492034 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:11.492252 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:11.492337 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:11.494853 (+ 2516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:11.494857 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:11.494859 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:11.494884 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:11.496156 (+ 1272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:11.496557 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:11.496559 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:11.496560 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:11.496633 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:11.496819 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:11.496821 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:11.496822 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:11.496991 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:11.497094 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:11.497287 (+ 193us) server_negotiation.cc:299] Negotiation successful
0929 10:33:11.497409 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 10:33:11.500459 17423 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51582:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 10:33:11.503536 17423 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:33:11.517836 17927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:11.511790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.67:40743 (local address 127.0.0.1:59772)
0929 10:33:11.512655 (+ 865us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:11.512668 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:11.512749 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:11.513713 (+ 964us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:11.513716 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:11.513734 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:33:11.513830 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:11.513836 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:11.515279 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:11.515282 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:11.516304 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:11.516314 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:11.517278 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:11.517282 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:11.517402 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:11.517443 (+ 41us) client_negotiation.cc:769] Sending connection context
0929 10:33:11.517558 (+ 115us) client_negotiation.cc:240] Negotiation successful
0929 10:33:11.517676 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":801,"thread_start_us":107,"threads_started":1}
I20260929 10:33:11.518026 17924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:11.511704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.65:45387 (local address 127.0.0.1:44176)
0929 10:33:11.512157 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:11.512204 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:11.512329 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:11.513405 (+ 1076us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:11.513411 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:11.513435 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:33:11.513579 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:11.513587 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:11.514805 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:11.514808 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:11.516193 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:11.516203 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:11.517558 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:11.517562 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:11.517646 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:11.517664 (+ 18us) client_negotiation.cc:769] Sending connection context
0929 10:33:11.517754 (+ 90us) client_negotiation.cc:240] Negotiation successful
0929 10:33:11.517826 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":74,"threads_started":1}
I20260929 10:33:11.518018 17926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:11.511839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44176 (local address 127.15.82.65:45387)
0929 10:33:11.512980 (+ 1141us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:11.512994 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:11.513018 (+ 24us) server_negotiation.cc:407] Connection header received
0929 10:33:11.513096 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:11.513102 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:11.513265 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:11.513387 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:33:11.513712 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:11.514693 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:11.516299 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:11.517161 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:11.517307 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:11.517464 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:11.517792 (+ 328us) server_negotiation.cc:299] Negotiation successful
0929 10:33:11.517906 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1010,"thread_start_us":98,"threads_started":1}
I20260929 10:33:11.518599 17925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:11.512395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.66:43273 (local address 127.0.0.1:53748)
0929 10:33:11.512856 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:11.512873 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:11.512929 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:11.514252 (+ 1323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:11.514255 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:11.514273 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:33:11.514371 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:11.514376 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:11.515691 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:11.515694 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:11.516780 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:11.516790 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:11.518159 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:11.518165 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:11.518287 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:11.518312 (+ 25us) client_negotiation.cc:769] Sending connection context
0929 10:33:11.518383 (+ 71us) client_negotiation.cc:240] Negotiation successful
0929 10:33:11.518476 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":113,"threads_started":1}
I20260929 10:33:11.518610 17928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:11.512487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59772 (local address 127.15.82.67:40743)
0929 10:33:11.513270 (+ 783us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:11.513280 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:11.513301 (+ 21us) server_negotiation.cc:407] Connection header received
0929 10:33:11.513390 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:11.513395 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:11.513573 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:11.513718 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:33:11.513940 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:11.515149 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:11.516449 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:11.517161 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:11.517278 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:11.517482 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:11.518322 (+ 840us) server_negotiation.cc:299] Negotiation successful
0929 10:33:11.518472 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":691,"thread_start_us":116,"threads_started":1}
I20260929 10:33:11.518685 17929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:11.512815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53748 (local address 127.15.82.66:43273)
0929 10:33:11.513151 (+ 336us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:11.513162 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:11.513185 (+ 23us) server_negotiation.cc:407] Connection header received
0929 10:33:11.513921 (+ 736us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:11.513927 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:11.514106 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:11.514256 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:33:11.514516 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:11.515594 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:11.516914 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:11.517788 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:11.517912 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:11.518086 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:11.518451 (+ 365us) server_negotiation.cc:299] Negotiation successful
0929 10:33:11.518548 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":230,"thread_start_us":112,"threads_started":1}
I20260929 10:33:11.521386 17701 tablet_service.cc:1467] Processing CreateTablet for tablet a2c82f9ce0b146e9b01d2ac56eab9d20 (DEFAULT_TABLE table=test-table [id=3e6861133e284c05b633cdd8887bac72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:33:11.521387 17561 tablet_service.cc:1467] Processing CreateTablet for tablet a2c82f9ce0b146e9b01d2ac56eab9d20 (DEFAULT_TABLE table=test-table [id=3e6861133e284c05b633cdd8887bac72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:33:11.521390 17841 tablet_service.cc:1467] Processing CreateTablet for tablet a2c82f9ce0b146e9b01d2ac56eab9d20 (DEFAULT_TABLE table=test-table [id=3e6861133e284c05b633cdd8887bac72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:33:11.522385 17701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2c82f9ce0b146e9b01d2ac56eab9d20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:33:11.522436 17561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2c82f9ce0b146e9b01d2ac56eab9d20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:33:11.522454 17841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2c82f9ce0b146e9b01d2ac56eab9d20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:33:11.529471 17930 tablet_bootstrap.cc:492] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 7f912be145e249078c19a61fb9f35c53: Bootstrap starting.
I20260929 10:33:11.530134 17932 tablet_bootstrap.cc:492] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 2b9cd0624fa84bc5b1a0de85bce66a72: Bootstrap starting.
I20260929 10:33:11.531913 17930 tablet_bootstrap.cc:654] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 7f912be145e249078c19a61fb9f35c53: Neither blocks nor log segments found. Creating new log.
I20260929 10:33:11.531926 17932 tablet_bootstrap.cc:654] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 2b9cd0624fa84bc5b1a0de85bce66a72: Neither blocks nor log segments found. Creating new log.
I20260929 10:33:11.532636 17930 log.cc:824] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 7f912be145e249078c19a61fb9f35c53: Log is configured to *not* fsync() on all Append() calls
I20260929 10:33:11.532636 17932 log.cc:824] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 2b9cd0624fa84bc5b1a0de85bce66a72: Log is configured to *not* fsync() on all Append() calls
I20260929 10:33:11.533296 17931 tablet_bootstrap.cc:492] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 1f1282b07b1f4d5c95e009991f21abbd: Bootstrap starting.
I20260929 10:33:11.534734 17932 tablet_bootstrap.cc:492] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 2b9cd0624fa84bc5b1a0de85bce66a72: No bootstrap required, opened a new log
I20260929 10:33:11.534929 17932 ts_tablet_manager.cc:1397] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 2b9cd0624fa84bc5b1a0de85bce66a72: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260929 10:33:11.535064 17930 tablet_bootstrap.cc:492] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 7f912be145e249078c19a61fb9f35c53: No bootstrap required, opened a new log
I20260929 10:33:11.535249 17930 ts_tablet_manager.cc:1397] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 7f912be145e249078c19a61fb9f35c53: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260929 10:33:11.535315 17931 tablet_bootstrap.cc:654] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 1f1282b07b1f4d5c95e009991f21abbd: Neither blocks nor log segments found. Creating new log.
I20260929 10:33:11.536278 17931 log.cc:824] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 1f1282b07b1f4d5c95e009991f21abbd: Log is configured to *not* fsync() on all Append() calls
I20260929 10:33:11.537863 17931 tablet_bootstrap.cc:492] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 1f1282b07b1f4d5c95e009991f21abbd: No bootstrap required, opened a new log
I20260929 10:33:11.538077 17931 ts_tablet_manager.cc:1397] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 1f1282b07b1f4d5c95e009991f21abbd: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260929 10:33:11.538964 17932 raft_consensus.cc:348] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 2b9cd0624fa84bc5b1a0de85bce66a72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1282b07b1f4d5c95e009991f21abbd" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45387 } } peers { permanent_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 40743 } } peers { permanent_uuid: "7f912be145e249078c19a61fb9f35c53" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43273 } }
I20260929 10:33:11.538964 17930 raft_consensus.cc:348] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 7f912be145e249078c19a61fb9f35c53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1282b07b1f4d5c95e009991f21abbd" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45387 } } peers { permanent_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 40743 } } peers { permanent_uuid: "7f912be145e249078c19a61fb9f35c53" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43273 } }
I20260929 10:33:11.539237 17932 raft_consensus.cc:374] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 2b9cd0624fa84bc5b1a0de85bce66a72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:33:11.539237 17930 raft_consensus.cc:374] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 7f912be145e249078c19a61fb9f35c53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:33:11.539296 17932 raft_consensus.cc:729] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 2b9cd0624fa84bc5b1a0de85bce66a72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b9cd0624fa84bc5b1a0de85bce66a72, State: Initialized, Role: FOLLOWER
I20260929 10:33:11.539296 17930 raft_consensus.cc:729] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 7f912be145e249078c19a61fb9f35c53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f912be145e249078c19a61fb9f35c53, State: Initialized, Role: FOLLOWER
I20260929 10:33:11.539757 17932 consensus_queue.cc:260] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 2b9cd0624fa84bc5b1a0de85bce66a72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1282b07b1f4d5c95e009991f21abbd" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45387 } } peers { permanent_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 40743 } } peers { permanent_uuid: "7f912be145e249078c19a61fb9f35c53" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43273 } }
I20260929 10:33:11.539757 17930 consensus_queue.cc:260] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 7f912be145e249078c19a61fb9f35c53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1282b07b1f4d5c95e009991f21abbd" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45387 } } peers { permanent_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 40743 } } peers { permanent_uuid: "7f912be145e249078c19a61fb9f35c53" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43273 } }
I20260929 10:33:11.540541 17767 heartbeater.cc:503] Master 127.15.82.126:39349 was elected leader, sending a full tablet report...
I20260929 10:33:11.540673 17907 heartbeater.cc:503] Master 127.15.82.126:39349 was elected leader, sending a full tablet report...
I20260929 10:33:11.540798 17932 ts_tablet_manager.cc:1428] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 2b9cd0624fa84bc5b1a0de85bce66a72: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260929 10:33:11.541651 17931 raft_consensus.cc:348] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 1f1282b07b1f4d5c95e009991f21abbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1282b07b1f4d5c95e009991f21abbd" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45387 } } peers { permanent_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 40743 } } peers { permanent_uuid: "7f912be145e249078c19a61fb9f35c53" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43273 } }
I20260929 10:33:11.541908 17931 raft_consensus.cc:374] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 1f1282b07b1f4d5c95e009991f21abbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:33:11.541957 17931 raft_consensus.cc:729] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 1f1282b07b1f4d5c95e009991f21abbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f1282b07b1f4d5c95e009991f21abbd, State: Initialized, Role: FOLLOWER
I20260929 10:33:11.542439 17931 consensus_queue.cc:260] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 1f1282b07b1f4d5c95e009991f21abbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1282b07b1f4d5c95e009991f21abbd" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45387 } } peers { permanent_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 40743 } } peers { permanent_uuid: "7f912be145e249078c19a61fb9f35c53" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43273 } }
I20260929 10:33:11.542728 17930 ts_tablet_manager.cc:1428] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 7f912be145e249078c19a61fb9f35c53: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260929 10:33:11.543409 17931 ts_tablet_manager.cc:1428] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 1f1282b07b1f4d5c95e009991f21abbd: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260929 10:33:11.543749 17627 heartbeater.cc:503] Master 127.15.82.126:39349 was elected leader, sending a full tablet report...
W20260929 10:33:11.708559 17908 tablet.cc:2369] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 2b9cd0624fa84bc5b1a0de85bce66a72: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 10:33:11.739918 17768 tablet.cc:2369] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 7f912be145e249078c19a61fb9f35c53: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 10:33:11.763351 17628 tablet.cc:2369] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 1f1282b07b1f4d5c95e009991f21abbd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 10:33:11.779320 17938 raft_consensus.cc:482] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 1f1282b07b1f4d5c95e009991f21abbd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:33:11.779525 17938 raft_consensus.cc:504] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 1f1282b07b1f4d5c95e009991f21abbd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1282b07b1f4d5c95e009991f21abbd" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45387 } } peers { permanent_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 40743 } } peers { permanent_uuid: "7f912be145e249078c19a61fb9f35c53" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43273 } }
I20260929 10:33:11.780710 17938 leader_election.cc:288] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 1f1282b07b1f4d5c95e009991f21abbd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b9cd0624fa84bc5b1a0de85bce66a72 (127.15.82.67:40743), 7f912be145e249078c19a61fb9f35c53 (127.15.82.66:43273)
I20260929 10:33:11.786255 17929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:11.781128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.65:60221 (local address 127.15.82.66:43273)
0929 10:33:11.781266 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:11.781271 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:11.781463 (+ 192us) server_negotiation.cc:407] Connection header received
0929 10:33:11.781596 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:11.781598 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:11.781642 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:11.781714 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:33:11.782224 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:11.783050 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:11.784945 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:11.785559 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:11.785685 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:11.785740 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:11.786046 (+ 306us) server_negotiation.cc:299] Negotiation successful
0929 10:33:11.786119 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260929 10:33:11.786257 17928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:11.781080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.65:45669 (local address 127.15.82.67:40743)
0929 10:33:11.781219 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:11.781225 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:11.781400 (+ 175us) server_negotiation.cc:407] Connection header received
0929 10:33:11.781566 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:11.781572 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:11.781628 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:11.781714 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:33:11.782224 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:11.783045 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:11.784945 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:11.785559 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:11.785685 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:11.785740 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:11.786046 (+ 306us) server_negotiation.cc:299] Negotiation successful
0929 10:33:11.786119 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260929 10:33:11.786317 17939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:11.780904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.67:40743 (local address 127.15.82.65:45669)
0929 10:33:11.781288 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:11.781304 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:11.781431 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:11.781751 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:11.781754 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:11.781793 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:33:11.782101 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:11.782108 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:11.783166 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:11.783170 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:11.784816 (+ 1646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:11.784828 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:11.785686 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:11.785690 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:11.785892 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:11.785933 (+ 41us) client_negotiation.cc:769] Sending connection context
0929 10:33:11.786013 (+ 80us) client_negotiation.cc:240] Negotiation successful
0929 10:33:11.786123 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"mutex_wait_us":360,"spinlock_wait_cycles":29952,"thread_start_us":127,"threads_started":1}
I20260929 10:33:11.786320 17940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:11.781050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.66:43273 (local address 127.15.82.65:60221)
0929 10:33:11.781388 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:11.781399 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:11.781495 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:11.781751 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:11.781754 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:11.781794 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:33:11.782101 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:11.782108 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:11.783166 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:11.783170 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:11.784816 (+ 1646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:11.784828 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:11.785686 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:11.785690 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:11.785886 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:11.785935 (+ 49us) client_negotiation.cc:769] Sending connection context
0929 10:33:11.786013 (+ 78us) client_negotiation.cc:240] Negotiation successful
0929 10:33:11.786120 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"mutex_wait_us":315,"spinlock_wait_cycles":9856,"thread_start_us":105,"threads_started":1}
I20260929 10:33:11.787043 17861 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2c82f9ce0b146e9b01d2ac56eab9d20" candidate_uuid: "1f1282b07b1f4d5c95e009991f21abbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" is_pre_election: true
I20260929 10:33:11.787169 17721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2c82f9ce0b146e9b01d2ac56eab9d20" candidate_uuid: "1f1282b07b1f4d5c95e009991f21abbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f912be145e249078c19a61fb9f35c53" is_pre_election: true
I20260929 10:33:11.787422 17721 raft_consensus.cc:2454] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 7f912be145e249078c19a61fb9f35c53 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f1282b07b1f4d5c95e009991f21abbd in term 0.
I20260929 10:33:11.787417 17861 raft_consensus.cc:2454] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 2b9cd0624fa84bc5b1a0de85bce66a72 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f1282b07b1f4d5c95e009991f21abbd in term 0.
I20260929 10:33:11.787968 17513 leader_election.cc:302] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 1f1282b07b1f4d5c95e009991f21abbd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f1282b07b1f4d5c95e009991f21abbd, 7f912be145e249078c19a61fb9f35c53; no voters:
I20260929 10:33:11.788256 17938 raft_consensus.cc:2781] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 1f1282b07b1f4d5c95e009991f21abbd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:33:11.788362 17938 raft_consensus.cc:482] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 1f1282b07b1f4d5c95e009991f21abbd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:33:11.788427 17938 raft_consensus.cc:3037] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 1f1282b07b1f4d5c95e009991f21abbd [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:33:11.789476 17938 raft_consensus.cc:504] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 1f1282b07b1f4d5c95e009991f21abbd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1282b07b1f4d5c95e009991f21abbd" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45387 } } peers { permanent_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 40743 } } peers { permanent_uuid: "7f912be145e249078c19a61fb9f35c53" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43273 } }
I20260929 10:33:11.789840 17938 leader_election.cc:288] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 1f1282b07b1f4d5c95e009991f21abbd [CANDIDATE]: Term 1 election: Requested vote from peers 2b9cd0624fa84bc5b1a0de85bce66a72 (127.15.82.67:40743), 7f912be145e249078c19a61fb9f35c53 (127.15.82.66:43273)
I20260929 10:33:11.790225 17721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2c82f9ce0b146e9b01d2ac56eab9d20" candidate_uuid: "1f1282b07b1f4d5c95e009991f21abbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f912be145e249078c19a61fb9f35c53"
I20260929 10:33:11.790374 17721 raft_consensus.cc:3037] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 7f912be145e249078c19a61fb9f35c53 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:33:11.790570 17861 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2c82f9ce0b146e9b01d2ac56eab9d20" candidate_uuid: "1f1282b07b1f4d5c95e009991f21abbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72"
I20260929 10:33:11.790702 17861 raft_consensus.cc:3037] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 2b9cd0624fa84bc5b1a0de85bce66a72 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:33:11.791610 17721 raft_consensus.cc:2454] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 7f912be145e249078c19a61fb9f35c53 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f1282b07b1f4d5c95e009991f21abbd in term 1.
I20260929 10:33:11.791967 17513 leader_election.cc:302] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 1f1282b07b1f4d5c95e009991f21abbd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f1282b07b1f4d5c95e009991f21abbd, 7f912be145e249078c19a61fb9f35c53; no voters:
I20260929 10:33:11.792143 17938 raft_consensus.cc:2781] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 1f1282b07b1f4d5c95e009991f21abbd [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:33:11.792294 17861 raft_consensus.cc:2454] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 2b9cd0624fa84bc5b1a0de85bce66a72 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f1282b07b1f4d5c95e009991f21abbd in term 1.
I20260929 10:33:11.792380 17938 raft_consensus.cc:686] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 1f1282b07b1f4d5c95e009991f21abbd [term 1 LEADER]: Becoming Leader. State: Replica: 1f1282b07b1f4d5c95e009991f21abbd, State: Running, Role: LEADER
I20260929 10:33:11.792758 17938 consensus_queue.cc:237] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 1f1282b07b1f4d5c95e009991f21abbd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1282b07b1f4d5c95e009991f21abbd" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45387 } } peers { permanent_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 40743 } } peers { permanent_uuid: "7f912be145e249078c19a61fb9f35c53" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43273 } }
I20260929 10:33:11.795791 17424 catalog_manager.cc:5495] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 1f1282b07b1f4d5c95e009991f21abbd reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f1282b07b1f4d5c95e009991f21abbd (127.15.82.65). New cstate: current_term: 1 leader_uuid: "1f1282b07b1f4d5c95e009991f21abbd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1282b07b1f4d5c95e009991f21abbd" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 40743 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f912be145e249078c19a61fb9f35c53" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43273 } health_report { overall_health: UNKNOWN } } }
Sep 29 10:33:11 dist-test-slave-vg89 krb5kdc[17379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790677990, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.126@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:33:11.828199 17473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:11.820225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51590 (local address 127.15.82.126:39349)
0929 10:33:11.820361 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:11.820365 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:11.820581 (+ 216us) server_negotiation.cc:407] Connection header received
0929 10:33:11.820690 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:11.820692 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:11.820732 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:11.820811 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:11.821457 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:11.821973 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:11.822750 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:11.822960 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:11.823040 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:11.825333 (+ 2293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:11.825336 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:11.825338 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:11.825359 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:11.826707 (+ 1348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:11.827130 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:11.827132 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:11.827134 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:11.827207 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:11.827434 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:11.827436 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:11.827437 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:11.827606 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:11.827682 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:11.827943 (+ 261us) server_negotiation.cc:299] Negotiation successful
0929 10:33:11.828065 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":55}
I20260929 10:33:11.828202 17948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:11.820182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:39349 (local address 127.0.0.1:51590)
0929 10:33:11.820479 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:11.820498 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:11.820609 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:11.820848 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:11.820850 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:11.821257 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:11.821352 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:11.821358 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:11.822067 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:11.822069 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:11.822650 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:11.822658 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:11.823058 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:11.823060 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:11.823119 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:11.823725 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:11.823737 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:11.825206 (+ 1469us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:11.826801 (+ 1595us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:11.826809 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:11.826814 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:11.827047 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:11.827283 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:11.827285 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:11.827286 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:11.827361 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:11.827693 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:11.827698 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:11.827800 (+ 102us) client_negotiation.cc:769] Sending connection context
0929 10:33:11.827889 (+ 89us) client_negotiation.cc:240] Negotiation successful
0929 10:33:11.828011 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":114,"threads_started":1}
I20260929 10:33:11.832674 17424 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.15.82.126@KRBTEST.COM'} at 127.0.0.1:51590:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260929 10:33:11.833422 17424 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:33:11.838284 17701 tablet_service.cc:1467] Processing CreateTablet for tablet 94aee3477c60439d9a8627408b9fe0cc (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=171af856da814fdc96bf1365afd3ad9b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 10:33:11.838271 17561 tablet_service.cc:1467] Processing CreateTablet for tablet 94aee3477c60439d9a8627408b9fe0cc (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=171af856da814fdc96bf1365afd3ad9b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 10:33:11.838611 17841 tablet_service.cc:1467] Processing CreateTablet for tablet 94aee3477c60439d9a8627408b9fe0cc (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=171af856da814fdc96bf1365afd3ad9b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 10:33:11.839056 17701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94aee3477c60439d9a8627408b9fe0cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:33:11.839061 17561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94aee3477c60439d9a8627408b9fe0cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:33:11.839154 17841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94aee3477c60439d9a8627408b9fe0cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:33:11.841686 17930 tablet_bootstrap.cc:492] T 94aee3477c60439d9a8627408b9fe0cc P 7f912be145e249078c19a61fb9f35c53: Bootstrap starting.
I20260929 10:33:11.842095 17931 tablet_bootstrap.cc:492] T 94aee3477c60439d9a8627408b9fe0cc P 1f1282b07b1f4d5c95e009991f21abbd: Bootstrap starting.
I20260929 10:33:11.842515 17932 tablet_bootstrap.cc:492] T 94aee3477c60439d9a8627408b9fe0cc P 2b9cd0624fa84bc5b1a0de85bce66a72: Bootstrap starting.
I20260929 10:33:11.842752 17930 tablet_bootstrap.cc:654] T 94aee3477c60439d9a8627408b9fe0cc P 7f912be145e249078c19a61fb9f35c53: Neither blocks nor log segments found. Creating new log.
I20260929 10:33:11.843170 17931 tablet_bootstrap.cc:654] T 94aee3477c60439d9a8627408b9fe0cc P 1f1282b07b1f4d5c95e009991f21abbd: Neither blocks nor log segments found. Creating new log.
I20260929 10:33:11.843480 17932 tablet_bootstrap.cc:654] T 94aee3477c60439d9a8627408b9fe0cc P 2b9cd0624fa84bc5b1a0de85bce66a72: Neither blocks nor log segments found. Creating new log.
I20260929 10:33:11.844129 17930 tablet_bootstrap.cc:492] T 94aee3477c60439d9a8627408b9fe0cc P 7f912be145e249078c19a61fb9f35c53: No bootstrap required, opened a new log
I20260929 10:33:11.844210 17930 ts_tablet_manager.cc:1397] T 94aee3477c60439d9a8627408b9fe0cc P 7f912be145e249078c19a61fb9f35c53: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260929 10:33:11.844769 17932 tablet_bootstrap.cc:492] T 94aee3477c60439d9a8627408b9fe0cc P 2b9cd0624fa84bc5b1a0de85bce66a72: No bootstrap required, opened a new log
I20260929 10:33:11.844728 17930 raft_consensus.cc:348] T 94aee3477c60439d9a8627408b9fe0cc P 7f912be145e249078c19a61fb9f35c53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1282b07b1f4d5c95e009991f21abbd" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45387 } } peers { permanent_uuid: "7f912be145e249078c19a61fb9f35c53" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43273 } } peers { permanent_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 40743 } }
I20260929 10:33:11.844856 17930 raft_consensus.cc:374] T 94aee3477c60439d9a8627408b9fe0cc P 7f912be145e249078c19a61fb9f35c53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:33:11.844859 17932 ts_tablet_manager.cc:1397] T 94aee3477c60439d9a8627408b9fe0cc P 2b9cd0624fa84bc5b1a0de85bce66a72: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 10:33:11.844899 17930 raft_consensus.cc:729] T 94aee3477c60439d9a8627408b9fe0cc P 7f912be145e249078c19a61fb9f35c53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f912be145e249078c19a61fb9f35c53, State: Initialized, Role: FOLLOWER
I20260929 10:33:11.845031 17930 consensus_queue.cc:260] T 94aee3477c60439d9a8627408b9fe0cc P 7f912be145e249078c19a61fb9f35c53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1282b07b1f4d5c95e009991f21abbd" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45387 } } peers { permanent_uuid: "7f912be145e249078c19a61fb9f35c53" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43273 } } peers { permanent_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 40743 } }
I20260929 10:33:11.845311 17930 ts_tablet_manager.cc:1428] T 94aee3477c60439d9a8627408b9fe0cc P 7f912be145e249078c19a61fb9f35c53: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260929 10:33:11.845314 17932 raft_consensus.cc:348] T 94aee3477c60439d9a8627408b9fe0cc P 2b9cd0624fa84bc5b1a0de85bce66a72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1282b07b1f4d5c95e009991f21abbd" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45387 } } peers { permanent_uuid: "7f912be145e249078c19a61fb9f35c53" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43273 } } peers { permanent_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 40743 } }
I20260929 10:33:11.845340 17937 tablet_replica.cc:439] T 94aee3477c60439d9a8627408b9fe0cc P 7f912be145e249078c19a61fb9f35c53: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1282b07b1f4d5c95e009991f21abbd" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45387 } } peers { permanent_uuid: "7f912be145e249078c19a61fb9f35c53" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43273 } } peers { permanent_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 40743 } } }
I20260929 10:33:11.845413 17932 raft_consensus.cc:374] T 94aee3477c60439d9a8627408b9fe0cc P 2b9cd0624fa84bc5b1a0de85bce66a72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:33:11.845451 17932 raft_consensus.cc:729] T 94aee3477c60439d9a8627408b9fe0cc P 2b9cd0624fa84bc5b1a0de85bce66a72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b9cd0624fa84bc5b1a0de85bce66a72, State: Initialized, Role: FOLLOWER
I20260929 10:33:11.845479 17937 tablet_replica.cc:442] T 94aee3477c60439d9a8627408b9fe0cc P 7f912be145e249078c19a61fb9f35c53: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 10:33:11.845588 17932 consensus_queue.cc:260] T 94aee3477c60439d9a8627408b9fe0cc P 2b9cd0624fa84bc5b1a0de85bce66a72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1282b07b1f4d5c95e009991f21abbd" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45387 } } peers { permanent_uuid: "7f912be145e249078c19a61fb9f35c53" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43273 } } peers { permanent_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 40743 } }
I20260929 10:33:11.845891 17932 ts_tablet_manager.cc:1428] T 94aee3477c60439d9a8627408b9fe0cc P 2b9cd0624fa84bc5b1a0de85bce66a72: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260929 10:33:11.845876 17936 tablet_replica.cc:439] T 94aee3477c60439d9a8627408b9fe0cc P 2b9cd0624fa84bc5b1a0de85bce66a72: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1282b07b1f4d5c95e009991f21abbd" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45387 } } peers { permanent_uuid: "7f912be145e249078c19a61fb9f35c53" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43273 } } peers { permanent_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 40743 } } }
I20260929 10:33:11.846040 17936 tablet_replica.cc:442] T 94aee3477c60439d9a8627408b9fe0cc P 2b9cd0624fa84bc5b1a0de85bce66a72: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 10:33:11.846626 17931 tablet_bootstrap.cc:492] T 94aee3477c60439d9a8627408b9fe0cc P 1f1282b07b1f4d5c95e009991f21abbd: No bootstrap required, opened a new log
I20260929 10:33:11.846727 17931 ts_tablet_manager.cc:1397] T 94aee3477c60439d9a8627408b9fe0cc P 1f1282b07b1f4d5c95e009991f21abbd: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260929 10:33:11.847417 17931 raft_consensus.cc:348] T 94aee3477c60439d9a8627408b9fe0cc P 1f1282b07b1f4d5c95e009991f21abbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1282b07b1f4d5c95e009991f21abbd" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45387 } } peers { permanent_uuid: "7f912be145e249078c19a61fb9f35c53" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43273 } } peers { permanent_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 40743 } }
I20260929 10:33:11.847566 17931 raft_consensus.cc:374] T 94aee3477c60439d9a8627408b9fe0cc P 1f1282b07b1f4d5c95e009991f21abbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:33:11.847615 17931 raft_consensus.cc:729] T 94aee3477c60439d9a8627408b9fe0cc P 1f1282b07b1f4d5c95e009991f21abbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f1282b07b1f4d5c95e009991f21abbd, State: Initialized, Role: FOLLOWER
I20260929 10:33:11.847746 17931 consensus_queue.cc:260] T 94aee3477c60439d9a8627408b9fe0cc P 1f1282b07b1f4d5c95e009991f21abbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1282b07b1f4d5c95e009991f21abbd" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45387 } } peers { permanent_uuid: "7f912be145e249078c19a61fb9f35c53" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43273 } } peers { permanent_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 40743 } }
I20260929 10:33:11.848047 17931 ts_tablet_manager.cc:1428] T 94aee3477c60439d9a8627408b9fe0cc P 1f1282b07b1f4d5c95e009991f21abbd: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260929 10:33:11.848001 17938 tablet_replica.cc:439] T 94aee3477c60439d9a8627408b9fe0cc P 1f1282b07b1f4d5c95e009991f21abbd: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1282b07b1f4d5c95e009991f21abbd" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45387 } } peers { permanent_uuid: "7f912be145e249078c19a61fb9f35c53" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43273 } } peers { permanent_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 40743 } } }
I20260929 10:33:11.848122 17938 tablet_replica.cc:442] T 94aee3477c60439d9a8627408b9fe0cc P 1f1282b07b1f4d5c95e009991f21abbd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 10:33:12.033948 17936 raft_consensus.cc:482] T 94aee3477c60439d9a8627408b9fe0cc P 2b9cd0624fa84bc5b1a0de85bce66a72 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:33:12.034085 17936 raft_consensus.cc:504] T 94aee3477c60439d9a8627408b9fe0cc P 2b9cd0624fa84bc5b1a0de85bce66a72 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1282b07b1f4d5c95e009991f21abbd" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45387 } } peers { permanent_uuid: "7f912be145e249078c19a61fb9f35c53" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43273 } } peers { permanent_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 40743 } }
I20260929 10:33:12.035341 17936 leader_election.cc:288] T 94aee3477c60439d9a8627408b9fe0cc P 2b9cd0624fa84bc5b1a0de85bce66a72 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f1282b07b1f4d5c95e009991f21abbd (127.15.82.65:45387), 7f912be145e249078c19a61fb9f35c53 (127.15.82.66:43273)
I20260929 10:33:12.040578 17929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:12.036123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.67:41953 (local address 127.15.82.66:43273)
0929 10:33:12.036248 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:12.036252 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:12.036265 (+ 13us) server_negotiation.cc:407] Connection header received
0929 10:33:12.036369 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:12.036371 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:12.036408 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:12.036491 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:33:12.036713 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:12.037429 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:12.039230 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:12.039822 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:12.039957 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:12.040012 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:12.040359 (+ 347us) server_negotiation.cc:299] Negotiation successful
0929 10:33:12.040431 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260929 10:33:12.040581 17953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:12.035656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.67:59183 (local address 127.15.82.65:45387)
0929 10:33:12.035879 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:12.035882 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:12.035917 (+ 35us) server_negotiation.cc:407] Connection header received
0929 10:33:12.036052 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:12.036056 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:12.036112 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:12.036216 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:33:12.036541 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:12.037405 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:12.039230 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:12.039847 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:12.039957 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:12.040012 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:12.040359 (+ 347us) server_negotiation.cc:299] Negotiation successful
0929 10:33:12.040436 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":82,"threads_started":1}
I20260929 10:33:12.040632 17952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:12.035551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.65:45387 (local address 127.15.82.67:59183)
0929 10:33:12.035806 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:12.035823 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:12.035932 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:12.036230 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:12.036235 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:12.036260 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:33:12.036408 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:12.036414 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:12.037543 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:12.037547 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:12.039093 (+ 1546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:12.039105 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:12.039990 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:12.039993 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:12.040200 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:12.040248 (+ 48us) client_negotiation.cc:769] Sending connection context
0929 10:33:12.040328 (+ 80us) client_negotiation.cc:240] Negotiation successful
0929 10:33:12.040452 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"mutex_wait_us":213,"thread_start_us":101,"threads_started":1}
I20260929 10:33:12.040632 17954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:12.035977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.66:43273 (local address 127.15.82.67:41953)
0929 10:33:12.036192 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:12.036206 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:12.036273 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:12.036503 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:12.036508 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:12.036530 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:33:12.036608 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:12.036612 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:12.037543 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:12.037547 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:12.039093 (+ 1546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:12.039105 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:12.039966 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:12.039970 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:12.040185 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:12.040248 (+ 63us) client_negotiation.cc:769] Sending connection context
0929 10:33:12.040328 (+ 80us) client_negotiation.cc:240] Negotiation successful
0929 10:33:12.040452 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"mutex_wait_us":211,"thread_start_us":82,"threads_started":1}
I20260929 10:33:12.041361 17581 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94aee3477c60439d9a8627408b9fe0cc" candidate_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f1282b07b1f4d5c95e009991f21abbd" is_pre_election: true
I20260929 10:33:12.041415 17721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94aee3477c60439d9a8627408b9fe0cc" candidate_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f912be145e249078c19a61fb9f35c53" is_pre_election: true
I20260929 10:33:12.041553 17721 raft_consensus.cc:2454] T 94aee3477c60439d9a8627408b9fe0cc P 7f912be145e249078c19a61fb9f35c53 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b9cd0624fa84bc5b1a0de85bce66a72 in term 0.
I20260929 10:33:12.041617 17581 raft_consensus.cc:2454] T 94aee3477c60439d9a8627408b9fe0cc P 1f1282b07b1f4d5c95e009991f21abbd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b9cd0624fa84bc5b1a0de85bce66a72 in term 0.
I20260929 10:33:12.041985 17793 leader_election.cc:302] T 94aee3477c60439d9a8627408b9fe0cc P 2b9cd0624fa84bc5b1a0de85bce66a72 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b9cd0624fa84bc5b1a0de85bce66a72, 7f912be145e249078c19a61fb9f35c53; no voters:
I20260929 10:33:12.042224 17936 raft_consensus.cc:2781] T 94aee3477c60439d9a8627408b9fe0cc P 2b9cd0624fa84bc5b1a0de85bce66a72 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:33:12.042302 17936 raft_consensus.cc:482] T 94aee3477c60439d9a8627408b9fe0cc P 2b9cd0624fa84bc5b1a0de85bce66a72 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:33:12.042348 17936 raft_consensus.cc:3037] T 94aee3477c60439d9a8627408b9fe0cc P 2b9cd0624fa84bc5b1a0de85bce66a72 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:33:12.043330 17936 raft_consensus.cc:504] T 94aee3477c60439d9a8627408b9fe0cc P 2b9cd0624fa84bc5b1a0de85bce66a72 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1282b07b1f4d5c95e009991f21abbd" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45387 } } peers { permanent_uuid: "7f912be145e249078c19a61fb9f35c53" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43273 } } peers { permanent_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 40743 } }
I20260929 10:33:12.043711 17936 leader_election.cc:288] T 94aee3477c60439d9a8627408b9fe0cc P 2b9cd0624fa84bc5b1a0de85bce66a72 [CANDIDATE]: Term 1 election: Requested vote from peers 1f1282b07b1f4d5c95e009991f21abbd (127.15.82.65:45387), 7f912be145e249078c19a61fb9f35c53 (127.15.82.66:43273)
I20260929 10:33:12.044114 17721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94aee3477c60439d9a8627408b9fe0cc" candidate_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f912be145e249078c19a61fb9f35c53"
I20260929 10:33:12.044106 17581 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94aee3477c60439d9a8627408b9fe0cc" candidate_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f1282b07b1f4d5c95e009991f21abbd"
I20260929 10:33:12.044229 17581 raft_consensus.cc:3037] T 94aee3477c60439d9a8627408b9fe0cc P 1f1282b07b1f4d5c95e009991f21abbd [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:33:12.044229 17721 raft_consensus.cc:3037] T 94aee3477c60439d9a8627408b9fe0cc P 7f912be145e249078c19a61fb9f35c53 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:33:12.045367 17721 raft_consensus.cc:2454] T 94aee3477c60439d9a8627408b9fe0cc P 7f912be145e249078c19a61fb9f35c53 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b9cd0624fa84bc5b1a0de85bce66a72 in term 1.
I20260929 10:33:12.045367 17581 raft_consensus.cc:2454] T 94aee3477c60439d9a8627408b9fe0cc P 1f1282b07b1f4d5c95e009991f21abbd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b9cd0624fa84bc5b1a0de85bce66a72 in term 1.
I20260929 10:33:12.045629 17793 leader_election.cc:302] T 94aee3477c60439d9a8627408b9fe0cc P 2b9cd0624fa84bc5b1a0de85bce66a72 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f1282b07b1f4d5c95e009991f21abbd, 2b9cd0624fa84bc5b1a0de85bce66a72; no voters:
I20260929 10:33:12.045791 17936 raft_consensus.cc:2781] T 94aee3477c60439d9a8627408b9fe0cc P 2b9cd0624fa84bc5b1a0de85bce66a72 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:33:12.045995 17936 raft_consensus.cc:686] T 94aee3477c60439d9a8627408b9fe0cc P 2b9cd0624fa84bc5b1a0de85bce66a72 [term 1 LEADER]: Becoming Leader. State: Replica: 2b9cd0624fa84bc5b1a0de85bce66a72, State: Running, Role: LEADER
I20260929 10:33:12.046345 17936 consensus_queue.cc:237] T 94aee3477c60439d9a8627408b9fe0cc P 2b9cd0624fa84bc5b1a0de85bce66a72 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1282b07b1f4d5c95e009991f21abbd" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45387 } } peers { permanent_uuid: "7f912be145e249078c19a61fb9f35c53" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43273 } } peers { permanent_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 40743 } }
I20260929 10:33:12.048676 17955 tablet_replica.cc:439] T 94aee3477c60439d9a8627408b9fe0cc P 2b9cd0624fa84bc5b1a0de85bce66a72: TxnStatusTablet state changed. Reason: New leader 2b9cd0624fa84bc5b1a0de85bce66a72. Latest consensus state: current_term: 1 leader_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1282b07b1f4d5c95e009991f21abbd" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45387 } } peers { permanent_uuid: "7f912be145e249078c19a61fb9f35c53" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43273 } } peers { permanent_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 40743 } } }
I20260929 10:33:12.048784 17955 tablet_replica.cc:442] T 94aee3477c60439d9a8627408b9fe0cc P 2b9cd0624fa84bc5b1a0de85bce66a72: This TxnStatusTablet replica's current role is: LEADER
I20260929 10:33:12.049218 17957 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260929 10:33:12.049429 17957 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260929 10:33:12.049652 17424 catalog_manager.cc:5495] T 94aee3477c60439d9a8627408b9fe0cc P 2b9cd0624fa84bc5b1a0de85bce66a72 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b9cd0624fa84bc5b1a0de85bce66a72 (127.15.82.67). New cstate: current_term: 1 leader_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1282b07b1f4d5c95e009991f21abbd" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f912be145e249078c19a61fb9f35c53" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 40743 } health_report { overall_health: HEALTHY } } }
I20260929 10:33:12.085305 17928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:12.081287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59778 (local address 127.15.82.67:40743)
0929 10:33:12.081412 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:12.081416 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:12.081463 (+ 47us) server_negotiation.cc:407] Connection header received
0929 10:33:12.081679 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:12.081682 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:12.081736 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:12.081822 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:33:12.082078 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:12.082640 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:12.083898 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:12.084104 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:12.084176 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:12.084509 (+ 333us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:33:12.084927 (+ 418us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:33:12.085022 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:12.085125 (+ 103us) server_negotiation.cc:299] Negotiation successful
0929 10:33:12.085174 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260929 10:33:12.085333 17948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:12.081158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.67:40743 (local address 127.0.0.1:59778)
0929 10:33:12.081403 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:12.081418 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:12.081532 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:12.081861 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:12.081864 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:12.081871 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 10:33:12.081996 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:12.082002 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:12.082747 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:12.082750 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:12.083783 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:12.083794 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:12.084214 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:12.084217 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:12.084311 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:12.084365 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 10:33:12.085022 (+ 657us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 10:33:12.085036 (+ 14us) client_negotiation.cc:769] Sending connection context
0929 10:33:12.085109 (+ 73us) client_negotiation.cc:240] Negotiation successful
0929 10:33:12.085192 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143}
I20260929 10:33:12.090878 17581 raft_consensus.cc:1261] T 94aee3477c60439d9a8627408b9fe0cc P 1f1282b07b1f4d5c95e009991f21abbd [term 1 FOLLOWER]: Refusing update from remote peer 2b9cd0624fa84bc5b1a0de85bce66a72: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:33:12.090926 17721 raft_consensus.cc:1261] T 94aee3477c60439d9a8627408b9fe0cc P 7f912be145e249078c19a61fb9f35c53 [term 1 FOLLOWER]: Refusing update from remote peer 2b9cd0624fa84bc5b1a0de85bce66a72: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:33:12.091585 17955 consensus_queue.cc:1035] T 94aee3477c60439d9a8627408b9fe0cc P 2b9cd0624fa84bc5b1a0de85bce66a72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f912be145e249078c19a61fb9f35c53" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:33:12.091802 17936 consensus_queue.cc:1035] T 94aee3477c60439d9a8627408b9fe0cc P 2b9cd0624fa84bc5b1a0de85bce66a72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f1282b07b1f4d5c95e009991f21abbd" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:33:12.095362 17938 tablet_replica.cc:439] T 94aee3477c60439d9a8627408b9fe0cc P 1f1282b07b1f4d5c95e009991f21abbd: TxnStatusTablet state changed. Reason: New leader 2b9cd0624fa84bc5b1a0de85bce66a72. Latest consensus state: current_term: 1 leader_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1282b07b1f4d5c95e009991f21abbd" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45387 } } peers { permanent_uuid: "7f912be145e249078c19a61fb9f35c53" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43273 } } peers { permanent_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 40743 } } }
I20260929 10:33:12.095497 17938 tablet_replica.cc:442] T 94aee3477c60439d9a8627408b9fe0cc P 1f1282b07b1f4d5c95e009991f21abbd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 10:33:12.096249 17937 tablet_replica.cc:439] T 94aee3477c60439d9a8627408b9fe0cc P 7f912be145e249078c19a61fb9f35c53: TxnStatusTablet state changed. Reason: New leader 2b9cd0624fa84bc5b1a0de85bce66a72. Latest consensus state: current_term: 1 leader_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1282b07b1f4d5c95e009991f21abbd" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45387 } } peers { permanent_uuid: "7f912be145e249078c19a61fb9f35c53" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43273 } } peers { permanent_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 40743 } } }
I20260929 10:33:12.096369 17937 tablet_replica.cc:442] T 94aee3477c60439d9a8627408b9fe0cc P 7f912be145e249078c19a61fb9f35c53: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 10:33:12.098297 17955 tablet_replica.cc:439] T 94aee3477c60439d9a8627408b9fe0cc P 2b9cd0624fa84bc5b1a0de85bce66a72: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1282b07b1f4d5c95e009991f21abbd" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45387 } } peers { permanent_uuid: "7f912be145e249078c19a61fb9f35c53" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43273 } } peers { permanent_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 40743 } } }
I20260929 10:33:12.098430 17955 tablet_replica.cc:442] T 94aee3477c60439d9a8627408b9fe0cc P 2b9cd0624fa84bc5b1a0de85bce66a72: This TxnStatusTablet replica's current role is: LEADER
I20260929 10:33:12.099109 17960 mvcc.cc:205] Tried to move back new op lower bound from 7334617055597346816 to 7334617055425060864. Current Snapshot: MvccSnapshot[applied={T|T < 7334617055597346816}]
I20260929 10:33:12.101091 17938 tablet_replica.cc:439] T 94aee3477c60439d9a8627408b9fe0cc P 1f1282b07b1f4d5c95e009991f21abbd: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1282b07b1f4d5c95e009991f21abbd" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45387 } } peers { permanent_uuid: "7f912be145e249078c19a61fb9f35c53" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43273 } } peers { permanent_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 40743 } } }
I20260929 10:33:12.101213 17938 tablet_replica.cc:442] T 94aee3477c60439d9a8627408b9fe0cc P 1f1282b07b1f4d5c95e009991f21abbd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 10:33:12.101323 17962 mvcc.cc:205] Tried to move back new op lower bound from 7334617055597346816 to 7334617055425060864. Current Snapshot: MvccSnapshot[applied={T|T < 7334617055597346816}]
I20260929 10:33:12.101897 17936 tablet_replica.cc:439] T 94aee3477c60439d9a8627408b9fe0cc P 2b9cd0624fa84bc5b1a0de85bce66a72: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1282b07b1f4d5c95e009991f21abbd" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45387 } } peers { permanent_uuid: "7f912be145e249078c19a61fb9f35c53" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43273 } } peers { permanent_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 40743 } } }
I20260929 10:33:12.102025 17936 tablet_replica.cc:442] T 94aee3477c60439d9a8627408b9fe0cc P 2b9cd0624fa84bc5b1a0de85bce66a72: This TxnStatusTablet replica's current role is: LEADER
I20260929 10:33:12.104529 17961 mvcc.cc:205] Tried to move back new op lower bound from 7334617055597346816 to 7334617055425060864. Current Snapshot: MvccSnapshot[applied={T|T < 7334617055597346816}]
I20260929 10:33:12.105212 17937 tablet_replica.cc:439] T 94aee3477c60439d9a8627408b9fe0cc P 7f912be145e249078c19a61fb9f35c53: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1282b07b1f4d5c95e009991f21abbd" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45387 } } peers { permanent_uuid: "7f912be145e249078c19a61fb9f35c53" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43273 } } peers { permanent_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 40743 } } }
I20260929 10:33:12.105341 17937 tablet_replica.cc:442] T 94aee3477c60439d9a8627408b9fe0cc P 7f912be145e249078c19a61fb9f35c53: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 10:33:12.111186 17953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:12.107359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44180 (local address 127.15.82.65:45387)
0929 10:33:12.107638 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:12.107646 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:12.107682 (+ 36us) server_negotiation.cc:407] Connection header received
0929 10:33:12.107858 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:12.107861 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:12.107922 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:12.108030 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:33:12.108290 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:12.108813 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:12.109696 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:12.109931 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:12.110042 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:12.110322 (+ 280us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:33:12.110801 (+ 479us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:33:12.110926 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:12.111001 (+ 75us) server_negotiation.cc:299] Negotiation successful
0929 10:33:12.111055 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200}
I20260929 10:33:12.122118 17928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:12.118765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.65:43741 (local address 127.15.82.67:40743)
0929 10:33:12.118905 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:12.118910 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:12.119051 (+ 141us) server_negotiation.cc:407] Connection header received
0929 10:33:12.119192 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:12.119195 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:12.119246 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:12.119327 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:33:12.119601 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:12.120122 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:12.121026 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:12.121259 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:12.121373 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:12.121683 (+ 310us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:33:12.121760 (+ 77us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:33:12.121837 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:12.121930 (+ 93us) server_negotiation.cc:299] Negotiation successful
0929 10:33:12.121978 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260929 10:33:12.122136 17972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:12.118661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.67:40743 (local address 127.15.82.65:43741)
0929 10:33:12.118952 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:12.118979 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:12.119100 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:12.119347 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:12.119350 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:12.119357 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 10:33:12.119490 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:12.119495 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:12.120228 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:12.120235 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:12.120893 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:12.120902 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:12.121375 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:12.121379 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:12.121483 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:12.121525 (+ 42us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 10:33:12.121851 (+ 326us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 10:33:12.121861 (+ 10us) client_negotiation.cc:769] Sending connection context
0929 10:33:12.121920 (+ 59us) client_negotiation.cc:240] Negotiation successful
0929 10:33:12.122004 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":99,"threads_started":1}
I20260929 10:33:12.129750 17861 raft_consensus.cc:1261] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 2b9cd0624fa84bc5b1a0de85bce66a72 [term 1 FOLLOWER]: Refusing update from remote peer 1f1282b07b1f4d5c95e009991f21abbd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:33:12.130081 17721 raft_consensus.cc:1261] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 7f912be145e249078c19a61fb9f35c53 [term 1 FOLLOWER]: Refusing update from remote peer 1f1282b07b1f4d5c95e009991f21abbd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:33:12.130275 17941 consensus_queue.cc:1035] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 1f1282b07b1f4d5c95e009991f21abbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 40743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:33:12.130478 17938 consensus_queue.cc:1035] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 1f1282b07b1f4d5c95e009991f21abbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f912be145e249078c19a61fb9f35c53" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:33:12.153416 17953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:12.149665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.67:42071 (local address 127.15.82.65:45387)
0929 10:33:12.149799 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:12.149806 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:12.149826 (+ 20us) server_negotiation.cc:407] Connection header received
0929 10:33:12.149882 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:12.149887 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:12.149943 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:12.150015 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:33:12.150383 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:12.151053 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:12.152271 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:12.152481 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:12.152555 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:12.152914 (+ 359us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:33:12.153024 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:33:12.153116 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:12.153225 (+ 109us) server_negotiation.cc:299] Negotiation successful
0929 10:33:12.153275 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260929 10:33:12.153475 17978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:12.149282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.65:45387 (local address 127.15.82.67:42071)
0929 10:33:12.149645 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:12.149661 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:12.149765 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:12.150124 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:12.150127 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:12.150135 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 10:33:12.150258 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:12.150264 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:12.151177 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:12.151181 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:12.152149 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:12.152160 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:12.152583 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:12.152587 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:12.152689 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:12.152740 (+ 51us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 10:33:12.153123 (+ 383us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 10:33:12.153132 (+ 9us) client_negotiation.cc:769] Sending connection context
0929 10:33:12.153205 (+ 73us) client_negotiation.cc:240] Negotiation successful
0929 10:33:12.153284 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":107,"threads_started":1}
I20260929 10:33:12.182392 17423 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51582:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:33:12.182638 17423 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51582:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:33:12.185503 17423 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3639d14359334e7fbfb18a16004331c8: Sending DeleteTablet for 3 replicas of tablet a2c82f9ce0b146e9b01d2ac56eab9d20
I20260929 10:33:12.186338 17701 tablet_service.cc:1514] Processing DeleteTablet for tablet a2c82f9ce0b146e9b01d2ac56eab9d20 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:33:12 UTC) from {username='kudu', principal='kudu/127.15.82.126@KRBTEST.COM'} at 127.0.0.1:53748
I20260929 10:33:12.186564 17841 tablet_service.cc:1514] Processing DeleteTablet for tablet a2c82f9ce0b146e9b01d2ac56eab9d20 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:33:12 UTC) from {username='kudu', principal='kudu/127.15.82.126@KRBTEST.COM'} at 127.0.0.1:59772
I20260929 10:33:12.187177 17561 tablet_service.cc:1514] Processing DeleteTablet for tablet a2c82f9ce0b146e9b01d2ac56eab9d20 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:33:12 UTC) from {username='kudu', principal='kudu/127.15.82.126@KRBTEST.COM'} at 127.0.0.1:44176
I20260929 10:33:12.187691 17981 tablet_replica.cc:330] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 7f912be145e249078c19a61fb9f35c53: stopping tablet replica
I20260929 10:33:12.187793 17423 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51582:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 10:33:12.188238 17423 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:33:12.188271 17981 raft_consensus.cc:2229] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 7f912be145e249078c19a61fb9f35c53 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:33:12.188620 17981 raft_consensus.cc:2258] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 7f912be145e249078c19a61fb9f35c53 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:33:12.189378 17983 tablet_replica.cc:330] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 1f1282b07b1f4d5c95e009991f21abbd: stopping tablet replica
I20260929 10:33:12.189658 17983 raft_consensus.cc:2229] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 1f1282b07b1f4d5c95e009991f21abbd [term 1 LEADER]: Raft consensus shutting down.
I20260929 10:33:12.190065 17983 raft_consensus.cc:2258] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 1f1282b07b1f4d5c95e009991f21abbd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:33:12.191792 17981 ts_tablet_manager.cc:1905] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 7f912be145e249078c19a61fb9f35c53: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:33:12.194808 17561 tablet_service.cc:1467] Processing CreateTablet for tablet 9352bc48c30f4e3db10a8a4f45697319 (DEFAULT_TABLE table=test-table [id=c16ba1485a7c4285b5b4fcf594e1e00c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:33:12.194882 17982 tablet_replica.cc:330] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 2b9cd0624fa84bc5b1a0de85bce66a72: stopping tablet replica
I20260929 10:33:12.195094 17561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9352bc48c30f4e3db10a8a4f45697319. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:33:12.195173 17841 tablet_service.cc:1467] Processing CreateTablet for tablet 9352bc48c30f4e3db10a8a4f45697319 (DEFAULT_TABLE table=test-table [id=c16ba1485a7c4285b5b4fcf594e1e00c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:33:12.195478 17841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9352bc48c30f4e3db10a8a4f45697319. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:33:12.197130 17983 ts_tablet_manager.cc:1905] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 1f1282b07b1f4d5c95e009991f21abbd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:33:12.197597 17701 tablet_service.cc:1467] Processing CreateTablet for tablet 9352bc48c30f4e3db10a8a4f45697319 (DEFAULT_TABLE table=test-table [id=c16ba1485a7c4285b5b4fcf594e1e00c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:33:12.197826 17701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9352bc48c30f4e3db10a8a4f45697319. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:33:12.197932 17932 tablet_bootstrap.cc:492] T 9352bc48c30f4e3db10a8a4f45697319 P 2b9cd0624fa84bc5b1a0de85bce66a72: Bootstrap starting.
I20260929 10:33:12.199021 17932 tablet_bootstrap.cc:654] T 9352bc48c30f4e3db10a8a4f45697319 P 2b9cd0624fa84bc5b1a0de85bce66a72: Neither blocks nor log segments found. Creating new log.
I20260929 10:33:12.195173 17982 raft_consensus.cc:2229] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 2b9cd0624fa84bc5b1a0de85bce66a72 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:33:12.200004 17982 raft_consensus.cc:2258] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 2b9cd0624fa84bc5b1a0de85bce66a72 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:33:12.200160 17931 tablet_bootstrap.cc:492] T 9352bc48c30f4e3db10a8a4f45697319 P 1f1282b07b1f4d5c95e009991f21abbd: Bootstrap starting.
I20260929 10:33:12.201129 17931 tablet_bootstrap.cc:654] T 9352bc48c30f4e3db10a8a4f45697319 P 1f1282b07b1f4d5c95e009991f21abbd: Neither blocks nor log segments found. Creating new log.
I20260929 10:33:12.201733 17983 ts_tablet_manager.cc:1918] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 1f1282b07b1f4d5c95e009991f21abbd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 10:33:12.201818 17983 log.cc:1196] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 1f1282b07b1f4d5c95e009991f21abbd: Deleting WAL directory at /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/ts-0/wal/wals/a2c82f9ce0b146e9b01d2ac56eab9d20
I20260929 10:33:12.201825 17930 tablet_bootstrap.cc:492] T 9352bc48c30f4e3db10a8a4f45697319 P 7f912be145e249078c19a61fb9f35c53: Bootstrap starting.
I20260929 10:33:12.202167 17983 ts_tablet_manager.cc:1939] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 1f1282b07b1f4d5c95e009991f21abbd: Deleting consensus metadata
I20260929 10:33:12.203349 17409 catalog_manager.cc:4841] TS 1f1282b07b1f4d5c95e009991f21abbd (127.15.82.65:45387): tablet a2c82f9ce0b146e9b01d2ac56eab9d20 (table test-table [id=3e6861133e284c05b633cdd8887bac72]) successfully deleted
I20260929 10:33:12.203634 17930 tablet_bootstrap.cc:654] T 9352bc48c30f4e3db10a8a4f45697319 P 7f912be145e249078c19a61fb9f35c53: Neither blocks nor log segments found. Creating new log.
I20260929 10:33:12.203817 17982 ts_tablet_manager.cc:1905] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 2b9cd0624fa84bc5b1a0de85bce66a72: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:33:12.204869 17981 ts_tablet_manager.cc:1918] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 7f912be145e249078c19a61fb9f35c53: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 10:33:12.204978 17981 log.cc:1196] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 7f912be145e249078c19a61fb9f35c53: Deleting WAL directory at /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/ts-1/wal/wals/a2c82f9ce0b146e9b01d2ac56eab9d20
I20260929 10:33:12.205363 17981 ts_tablet_manager.cc:1939] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 7f912be145e249078c19a61fb9f35c53: Deleting consensus metadata
I20260929 10:33:12.206317 17982 ts_tablet_manager.cc:1918] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 2b9cd0624fa84bc5b1a0de85bce66a72: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 10:33:12.206423 17982 log.cc:1196] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 2b9cd0624fa84bc5b1a0de85bce66a72: Deleting WAL directory at /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/ts-2/wal/wals/a2c82f9ce0b146e9b01d2ac56eab9d20
I20260929 10:33:12.206549 17409 catalog_manager.cc:4841] TS 7f912be145e249078c19a61fb9f35c53 (127.15.82.66:43273): tablet a2c82f9ce0b146e9b01d2ac56eab9d20 (table test-table [id=3e6861133e284c05b633cdd8887bac72]) successfully deleted
I20260929 10:33:12.206758 17982 ts_tablet_manager.cc:1939] T a2c82f9ce0b146e9b01d2ac56eab9d20 P 2b9cd0624fa84bc5b1a0de85bce66a72: Deleting consensus metadata
I20260929 10:33:12.207232 17931 tablet_bootstrap.cc:492] T 9352bc48c30f4e3db10a8a4f45697319 P 1f1282b07b1f4d5c95e009991f21abbd: No bootstrap required, opened a new log
I20260929 10:33:12.207332 17931 ts_tablet_manager.cc:1397] T 9352bc48c30f4e3db10a8a4f45697319 P 1f1282b07b1f4d5c95e009991f21abbd: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260929 10:33:12.207757 17931 raft_consensus.cc:348] T 9352bc48c30f4e3db10a8a4f45697319 P 1f1282b07b1f4d5c95e009991f21abbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1282b07b1f4d5c95e009991f21abbd" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45387 } } peers { permanent_uuid: "7f912be145e249078c19a61fb9f35c53" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43273 } } peers { permanent_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 40743 } }
I20260929 10:33:12.207823 17411 catalog_manager.cc:4841] TS 2b9cd0624fa84bc5b1a0de85bce66a72 (127.15.82.67:40743): tablet a2c82f9ce0b146e9b01d2ac56eab9d20 (table test-table [id=3e6861133e284c05b633cdd8887bac72]) successfully deleted
I20260929 10:33:12.207873 17931 raft_consensus.cc:374] T 9352bc48c30f4e3db10a8a4f45697319 P 1f1282b07b1f4d5c95e009991f21abbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:33:12.207904 17931 raft_consensus.cc:729] T 9352bc48c30f4e3db10a8a4f45697319 P 1f1282b07b1f4d5c95e009991f21abbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f1282b07b1f4d5c95e009991f21abbd, State: Initialized, Role: FOLLOWER
I20260929 10:33:12.208040 17931 consensus_queue.cc:260] T 9352bc48c30f4e3db10a8a4f45697319 P 1f1282b07b1f4d5c95e009991f21abbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1282b07b1f4d5c95e009991f21abbd" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45387 } } peers { permanent_uuid: "7f912be145e249078c19a61fb9f35c53" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43273 } } peers { permanent_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 40743 } }
I20260929 10:33:12.208315 17931 ts_tablet_manager.cc:1428] T 9352bc48c30f4e3db10a8a4f45697319 P 1f1282b07b1f4d5c95e009991f21abbd: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260929 10:33:12.208570 17932 tablet_bootstrap.cc:492] T 9352bc48c30f4e3db10a8a4f45697319 P 2b9cd0624fa84bc5b1a0de85bce66a72: No bootstrap required, opened a new log
I20260929 10:33:12.208671 17932 ts_tablet_manager.cc:1397] T 9352bc48c30f4e3db10a8a4f45697319 P 2b9cd0624fa84bc5b1a0de85bce66a72: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20260929 10:33:12.209056 17930 tablet_bootstrap.cc:492] T 9352bc48c30f4e3db10a8a4f45697319 P 7f912be145e249078c19a61fb9f35c53: No bootstrap required, opened a new log
I20260929 10:33:12.209126 17930 ts_tablet_manager.cc:1397] T 9352bc48c30f4e3db10a8a4f45697319 P 7f912be145e249078c19a61fb9f35c53: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260929 10:33:12.209100 17932 raft_consensus.cc:348] T 9352bc48c30f4e3db10a8a4f45697319 P 2b9cd0624fa84bc5b1a0de85bce66a72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1282b07b1f4d5c95e009991f21abbd" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45387 } } peers { permanent_uuid: "7f912be145e249078c19a61fb9f35c53" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43273 } } peers { permanent_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 40743 } }
I20260929 10:33:12.209203 17932 raft_consensus.cc:374] T 9352bc48c30f4e3db10a8a4f45697319 P 2b9cd0624fa84bc5b1a0de85bce66a72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:33:12.209244 17932 raft_consensus.cc:729] T 9352bc48c30f4e3db10a8a4f45697319 P 2b9cd0624fa84bc5b1a0de85bce66a72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b9cd0624fa84bc5b1a0de85bce66a72, State: Initialized, Role: FOLLOWER
I20260929 10:33:12.209375 17932 consensus_queue.cc:260] T 9352bc48c30f4e3db10a8a4f45697319 P 2b9cd0624fa84bc5b1a0de85bce66a72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1282b07b1f4d5c95e009991f21abbd" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45387 } } peers { permanent_uuid: "7f912be145e249078c19a61fb9f35c53" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43273 } } peers { permanent_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 40743 } }
I20260929 10:33:12.209517 17930 raft_consensus.cc:348] T 9352bc48c30f4e3db10a8a4f45697319 P 7f912be145e249078c19a61fb9f35c53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1282b07b1f4d5c95e009991f21abbd" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45387 } } peers { permanent_uuid: "7f912be145e249078c19a61fb9f35c53" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43273 } } peers { permanent_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 40743 } }
I20260929 10:33:12.209630 17930 raft_consensus.cc:374] T 9352bc48c30f4e3db10a8a4f45697319 P 7f912be145e249078c19a61fb9f35c53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:33:12.209661 17930 raft_consensus.cc:729] T 9352bc48c30f4e3db10a8a4f45697319 P 7f912be145e249078c19a61fb9f35c53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f912be145e249078c19a61fb9f35c53, State: Initialized, Role: FOLLOWER
I20260929 10:33:12.209669 17932 ts_tablet_manager.cc:1428] T 9352bc48c30f4e3db10a8a4f45697319 P 2b9cd0624fa84bc5b1a0de85bce66a72: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260929 10:33:12.209806 17930 consensus_queue.cc:260] T 9352bc48c30f4e3db10a8a4f45697319 P 7f912be145e249078c19a61fb9f35c53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1282b07b1f4d5c95e009991f21abbd" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45387 } } peers { permanent_uuid: "7f912be145e249078c19a61fb9f35c53" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43273 } } peers { permanent_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 40743 } }
I20260929 10:33:12.210106 17930 ts_tablet_manager.cc:1428] T 9352bc48c30f4e3db10a8a4f45697319 P 7f912be145e249078c19a61fb9f35c53: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260929 10:33:12.245592 17937 raft_consensus.cc:482] T 9352bc48c30f4e3db10a8a4f45697319 P 7f912be145e249078c19a61fb9f35c53 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:33:12.245755 17937 raft_consensus.cc:504] T 9352bc48c30f4e3db10a8a4f45697319 P 7f912be145e249078c19a61fb9f35c53 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1282b07b1f4d5c95e009991f21abbd" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45387 } } peers { permanent_uuid: "7f912be145e249078c19a61fb9f35c53" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43273 } } peers { permanent_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 40743 } }
I20260929 10:33:12.246928 17937 leader_election.cc:288] T 9352bc48c30f4e3db10a8a4f45697319 P 7f912be145e249078c19a61fb9f35c53 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f1282b07b1f4d5c95e009991f21abbd (127.15.82.65:45387), 2b9cd0624fa84bc5b1a0de85bce66a72 (127.15.82.67:40743)
I20260929 10:33:12.252523 17953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:12.247582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:43557 (local address 127.15.82.65:45387)
0929 10:33:12.247698 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:12.247702 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:12.247949 (+ 247us) server_negotiation.cc:407] Connection header received
0929 10:33:12.248082 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:12.248085 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:12.248139 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:12.248209 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:33:12.248624 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:12.249418 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:12.251089 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:12.251708 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:12.251788 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:12.251833 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:12.252308 (+ 475us) server_negotiation.cc:299] Negotiation successful
0929 10:33:12.252395 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260929 10:33:12.252523 17928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:12.247573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:58591 (local address 127.15.82.67:40743)
0929 10:33:12.247698 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:12.247702 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:12.248276 (+ 574us) server_negotiation.cc:407] Connection header received
0929 10:33:12.248449 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:12.248452 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:12.248501 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:12.248551 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:33:12.248934 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:12.249620 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:12.251089 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:12.251684 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:12.251781 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:12.251833 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:12.252309 (+ 476us) server_negotiation.cc:299] Negotiation successful
0929 10:33:12.252392 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260929 10:33:12.253018 17988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:12.247183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.67:40743 (local address 127.15.82.66:58591)
0929 10:33:12.248199 (+ 1016us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:12.248212 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:12.248333 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:12.248683 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:12.248687 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:12.248701 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:33:12.248800 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:12.248806 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:12.249737 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:12.249740 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:12.250946 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:12.250956 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:12.251931 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:12.251935 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:12.252128 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:12.252185 (+ 57us) client_negotiation.cc:769] Sending connection context
0929 10:33:12.252789 (+ 604us) client_negotiation.cc:240] Negotiation successful
0929 10:33:12.252873 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":941,"mutex_wait_us":104,"thread_start_us":70,"threads_started":1}
I20260929 10:33:12.253309 17987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:12.247099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.65:45387 (local address 127.15.82.66:43557)
0929 10:33:12.247843 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:12.247861 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:12.247987 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:12.248329 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:12.248332 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:12.248345 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:33:12.248499 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:12.248505 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:12.249549 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:12.249552 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:12.250948 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:12.250956 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:12.251931 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:12.251935 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:12.252160 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:12.252190 (+ 30us) client_negotiation.cc:769] Sending connection context
0929 10:33:12.252964 (+ 774us) client_negotiation.cc:240] Negotiation successful
0929 10:33:12.253027 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":684,"mutex_wait_us":96,"thread_start_us":132,"threads_started":1}
I20260929 10:33:12.253463 17861 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9352bc48c30f4e3db10a8a4f45697319" candidate_uuid: "7f912be145e249078c19a61fb9f35c53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" is_pre_election: true
I20260929 10:33:12.253620 17861 raft_consensus.cc:2454] T 9352bc48c30f4e3db10a8a4f45697319 P 2b9cd0624fa84bc5b1a0de85bce66a72 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f912be145e249078c19a61fb9f35c53 in term 0.
I20260929 10:33:12.254065 17581 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9352bc48c30f4e3db10a8a4f45697319" candidate_uuid: "7f912be145e249078c19a61fb9f35c53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f1282b07b1f4d5c95e009991f21abbd" is_pre_election: true
I20260929 10:33:12.254206 17581 raft_consensus.cc:2454] T 9352bc48c30f4e3db10a8a4f45697319 P 1f1282b07b1f4d5c95e009991f21abbd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f912be145e249078c19a61fb9f35c53 in term 0.
I20260929 10:33:12.254261 17655 leader_election.cc:302] T 9352bc48c30f4e3db10a8a4f45697319 P 7f912be145e249078c19a61fb9f35c53 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b9cd0624fa84bc5b1a0de85bce66a72, 7f912be145e249078c19a61fb9f35c53; no voters:
I20260929 10:33:12.254560 17937 raft_consensus.cc:2781] T 9352bc48c30f4e3db10a8a4f45697319 P 7f912be145e249078c19a61fb9f35c53 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:33:12.254623 17937 raft_consensus.cc:482] T 9352bc48c30f4e3db10a8a4f45697319 P 7f912be145e249078c19a61fb9f35c53 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:33:12.254671 17937 raft_consensus.cc:3037] T 9352bc48c30f4e3db10a8a4f45697319 P 7f912be145e249078c19a61fb9f35c53 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:33:12.255661 17937 raft_consensus.cc:504] T 9352bc48c30f4e3db10a8a4f45697319 P 7f912be145e249078c19a61fb9f35c53 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1282b07b1f4d5c95e009991f21abbd" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45387 } } peers { permanent_uuid: "7f912be145e249078c19a61fb9f35c53" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43273 } } peers { permanent_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 40743 } }
I20260929 10:33:12.256105 17937 leader_election.cc:288] T 9352bc48c30f4e3db10a8a4f45697319 P 7f912be145e249078c19a61fb9f35c53 [CANDIDATE]: Term 1 election: Requested vote from peers 1f1282b07b1f4d5c95e009991f21abbd (127.15.82.65:45387), 2b9cd0624fa84bc5b1a0de85bce66a72 (127.15.82.67:40743)
I20260929 10:33:12.256464 17581 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9352bc48c30f4e3db10a8a4f45697319" candidate_uuid: "7f912be145e249078c19a61fb9f35c53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f1282b07b1f4d5c95e009991f21abbd"
I20260929 10:33:12.256578 17581 raft_consensus.cc:3037] T 9352bc48c30f4e3db10a8a4f45697319 P 1f1282b07b1f4d5c95e009991f21abbd [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:33:12.257632 17581 raft_consensus.cc:2454] T 9352bc48c30f4e3db10a8a4f45697319 P 1f1282b07b1f4d5c95e009991f21abbd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f912be145e249078c19a61fb9f35c53 in term 1.
I20260929 10:33:12.258277 17861 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9352bc48c30f4e3db10a8a4f45697319" candidate_uuid: "7f912be145e249078c19a61fb9f35c53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72"
I20260929 10:33:12.258424 17653 leader_election.cc:302] T 9352bc48c30f4e3db10a8a4f45697319 P 7f912be145e249078c19a61fb9f35c53 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f1282b07b1f4d5c95e009991f21abbd, 7f912be145e249078c19a61fb9f35c53; no voters:
I20260929 10:33:12.258453 17861 raft_consensus.cc:3037] T 9352bc48c30f4e3db10a8a4f45697319 P 2b9cd0624fa84bc5b1a0de85bce66a72 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:33:12.258669 17937 raft_consensus.cc:2781] T 9352bc48c30f4e3db10a8a4f45697319 P 7f912be145e249078c19a61fb9f35c53 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:33:12.258891 17937 raft_consensus.cc:686] T 9352bc48c30f4e3db10a8a4f45697319 P 7f912be145e249078c19a61fb9f35c53 [term 1 LEADER]: Becoming Leader. State: Replica: 7f912be145e249078c19a61fb9f35c53, State: Running, Role: LEADER
I20260929 10:33:12.259148 17937 consensus_queue.cc:237] T 9352bc48c30f4e3db10a8a4f45697319 P 7f912be145e249078c19a61fb9f35c53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1282b07b1f4d5c95e009991f21abbd" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45387 } } peers { permanent_uuid: "7f912be145e249078c19a61fb9f35c53" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43273 } } peers { permanent_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 40743 } }
I20260929 10:33:12.259572 17861 raft_consensus.cc:2454] T 9352bc48c30f4e3db10a8a4f45697319 P 2b9cd0624fa84bc5b1a0de85bce66a72 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f912be145e249078c19a61fb9f35c53 in term 1.
I20260929 10:33:12.261615 17423 catalog_manager.cc:5495] T 9352bc48c30f4e3db10a8a4f45697319 P 7f912be145e249078c19a61fb9f35c53 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7f912be145e249078c19a61fb9f35c53 (127.15.82.66). New cstate: current_term: 1 leader_uuid: "7f912be145e249078c19a61fb9f35c53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1282b07b1f4d5c95e009991f21abbd" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f912be145e249078c19a61fb9f35c53" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43273 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 40743 } health_report { overall_health: UNKNOWN } } }
I20260929 10:33:12.290668 17929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:12.286870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53756 (local address 127.15.82.66:43273)
0929 10:33:12.286977 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:12.286981 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:12.287002 (+ 21us) server_negotiation.cc:407] Connection header received
0929 10:33:12.287192 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:12.287194 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:12.287247 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:12.287362 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:33:12.287591 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:12.288137 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:12.289219 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:12.289427 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:12.289500 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:12.289782 (+ 282us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:33:12.290174 (+ 392us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:33:12.290258 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:12.290370 (+ 112us) server_negotiation.cc:299] Negotiation successful
0929 10:33:12.290491 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260929 10:33:12.301693 17928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:12.297790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:33433 (local address 127.15.82.67:40743)
0929 10:33:12.297999 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:12.298002 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:12.298190 (+ 188us) server_negotiation.cc:407] Connection header received
0929 10:33:12.298250 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:12.298253 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:12.298314 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:12.298378 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:33:12.298720 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:12.299302 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:12.300640 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:12.300850 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:12.300927 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:12.301292 (+ 365us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:33:12.301358 (+ 66us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:33:12.301422 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:12.301535 (+ 113us) server_negotiation.cc:299] Negotiation successful
0929 10:33:12.301581 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123}
I20260929 10:33:12.301820 17993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:12.297638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.67:40743 (local address 127.15.82.66:33433)
0929 10:33:12.297989 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:12.298007 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:12.298126 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:12.298430 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:12.298433 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:12.298442 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 10:33:12.298598 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:12.298605 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:12.299413 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:12.299417 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:12.300525 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:12.300537 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:12.300980 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:12.300985 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:12.301100 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:12.301160 (+ 60us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 10:33:12.301442 (+ 282us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 10:33:12.301448 (+ 6us) client_negotiation.cc:769] Sending connection context
0929 10:33:12.301517 (+ 69us) client_negotiation.cc:240] Negotiation successful
0929 10:33:12.301632 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":129,"threads_started":1}
I20260929 10:33:12.308084 17861 raft_consensus.cc:1261] T 9352bc48c30f4e3db10a8a4f45697319 P 2b9cd0624fa84bc5b1a0de85bce66a72 [term 1 FOLLOWER]: Refusing update from remote peer 7f912be145e249078c19a61fb9f35c53: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:33:12.308475 17581 raft_consensus.cc:1261] T 9352bc48c30f4e3db10a8a4f45697319 P 1f1282b07b1f4d5c95e009991f21abbd [term 1 FOLLOWER]: Refusing update from remote peer 7f912be145e249078c19a61fb9f35c53: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:33:12.308620 17937 consensus_queue.cc:1035] T 9352bc48c30f4e3db10a8a4f45697319 P 7f912be145e249078c19a61fb9f35c53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 40743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:33:12.308857 17989 consensus_queue.cc:1035] T 9352bc48c30f4e3db10a8a4f45697319 P 7f912be145e249078c19a61fb9f35c53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f1282b07b1f4d5c95e009991f21abbd" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:33:12.329248 17929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:12.325649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.67:51981 (local address 127.15.82.66:43273)
0929 10:33:12.325755 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:12.325760 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:12.325804 (+ 44us) server_negotiation.cc:407] Connection header received
0929 10:33:12.325992 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:12.325996 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:12.326044 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:12.326120 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:33:12.326368 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:12.327172 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:12.328063 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:12.328385 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:12.328503 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:12.328780 (+ 277us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:33:12.328865 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:33:12.328944 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:12.329063 (+ 119us) server_negotiation.cc:299] Negotiation successful
0929 10:33:12.329111 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260929 10:33:12.329248 17978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:12.325607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.66:43273 (local address 127.15.82.67:51981)
0929 10:33:12.325739 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:12.325756 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:12.325854 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:12.326131 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:12.326134 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:12.326142 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 10:33:12.326251 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:12.326258 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:12.327281 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:12.327283 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:12.327947 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:12.327956 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:12.328510 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:12.328513 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:12.328588 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:12.328630 (+ 42us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 10:33:12.328965 (+ 335us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 10:33:12.328973 (+ 8us) client_negotiation.cc:769] Sending connection context
0929 10:33:12.329047 (+ 74us) client_negotiation.cc:240] Negotiation successful
0929 10:33:12.329130 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
W20260929 10:33:12.330024 17961 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260929 10:33:13.893555 17422 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51582:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:33:13.893703 17422 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51582:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:33:13.896199 17422 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3639d14359334e7fbfb18a16004331c8: Sending DeleteTablet for 3 replicas of tablet 9352bc48c30f4e3db10a8a4f45697319
I20260929 10:33:13.896728 17561 tablet_service.cc:1514] Processing DeleteTablet for tablet 9352bc48c30f4e3db10a8a4f45697319 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:33:13 UTC) from {username='kudu', principal='kudu/127.15.82.126@KRBTEST.COM'} at 127.0.0.1:44176
I20260929 10:33:13.896786 17841 tablet_service.cc:1514] Processing DeleteTablet for tablet 9352bc48c30f4e3db10a8a4f45697319 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:33:13 UTC) from {username='kudu', principal='kudu/127.15.82.126@KRBTEST.COM'} at 127.0.0.1:59772
I20260929 10:33:13.896829 17701 tablet_service.cc:1514] Processing DeleteTablet for tablet 9352bc48c30f4e3db10a8a4f45697319 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:33:13 UTC) from {username='kudu', principal='kudu/127.15.82.126@KRBTEST.COM'} at 127.0.0.1:53748
I20260929 10:33:13.897091 18019 tablet_replica.cc:330] T 9352bc48c30f4e3db10a8a4f45697319 P 1f1282b07b1f4d5c95e009991f21abbd: stopping tablet replica
I20260929 10:33:13.897128 18021 tablet_replica.cc:330] T 9352bc48c30f4e3db10a8a4f45697319 P 2b9cd0624fa84bc5b1a0de85bce66a72: stopping tablet replica
I20260929 10:33:13.897228 18019 raft_consensus.cc:2229] T 9352bc48c30f4e3db10a8a4f45697319 P 1f1282b07b1f4d5c95e009991f21abbd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:33:13.897246 18021 raft_consensus.cc:2229] T 9352bc48c30f4e3db10a8a4f45697319 P 2b9cd0624fa84bc5b1a0de85bce66a72 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:33:13.897346 18019 raft_consensus.cc:2258] T 9352bc48c30f4e3db10a8a4f45697319 P 1f1282b07b1f4d5c95e009991f21abbd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:33:13.897363 18021 raft_consensus.cc:2258] T 9352bc48c30f4e3db10a8a4f45697319 P 2b9cd0624fa84bc5b1a0de85bce66a72 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:33:13.897746 18020 tablet_replica.cc:330] T 9352bc48c30f4e3db10a8a4f45697319 P 7f912be145e249078c19a61fb9f35c53: stopping tablet replica
I20260929 10:33:13.897868 18020 raft_consensus.cc:2229] T 9352bc48c30f4e3db10a8a4f45697319 P 7f912be145e249078c19a61fb9f35c53 [term 1 LEADER]: Raft consensus shutting down.
I20260929 10:33:13.898036 18021 ts_tablet_manager.cc:1905] T 9352bc48c30f4e3db10a8a4f45697319 P 2b9cd0624fa84bc5b1a0de85bce66a72: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:33:13.898074 18019 ts_tablet_manager.cc:1905] T 9352bc48c30f4e3db10a8a4f45697319 P 1f1282b07b1f4d5c95e009991f21abbd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:33:13.898173 18020 raft_consensus.cc:2258] T 9352bc48c30f4e3db10a8a4f45697319 P 7f912be145e249078c19a61fb9f35c53 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:33:13.898805 18020 ts_tablet_manager.cc:1905] T 9352bc48c30f4e3db10a8a4f45697319 P 7f912be145e249078c19a61fb9f35c53: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:33:13.900835 18021 ts_tablet_manager.cc:1918] T 9352bc48c30f4e3db10a8a4f45697319 P 2b9cd0624fa84bc5b1a0de85bce66a72: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260929 10:33:13.900914 18021 log.cc:1196] T 9352bc48c30f4e3db10a8a4f45697319 P 2b9cd0624fa84bc5b1a0de85bce66a72: Deleting WAL directory at /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/ts-2/wal/wals/9352bc48c30f4e3db10a8a4f45697319
I20260929 10:33:13.901191 18021 ts_tablet_manager.cc:1939] T 9352bc48c30f4e3db10a8a4f45697319 P 2b9cd0624fa84bc5b1a0de85bce66a72: Deleting consensus metadata
I20260929 10:33:13.901266 18019 ts_tablet_manager.cc:1918] T 9352bc48c30f4e3db10a8a4f45697319 P 1f1282b07b1f4d5c95e009991f21abbd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260929 10:33:13.901348 18019 log.cc:1196] T 9352bc48c30f4e3db10a8a4f45697319 P 1f1282b07b1f4d5c95e009991f21abbd: Deleting WAL directory at /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/ts-0/wal/wals/9352bc48c30f4e3db10a8a4f45697319
I20260929 10:33:13.901630 18019 ts_tablet_manager.cc:1939] T 9352bc48c30f4e3db10a8a4f45697319 P 1f1282b07b1f4d5c95e009991f21abbd: Deleting consensus metadata
I20260929 10:33:13.901722 18020 ts_tablet_manager.cc:1918] T 9352bc48c30f4e3db10a8a4f45697319 P 7f912be145e249078c19a61fb9f35c53: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260929 10:33:13.901803 18020 log.cc:1196] T 9352bc48c30f4e3db10a8a4f45697319 P 7f912be145e249078c19a61fb9f35c53: Deleting WAL directory at /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/ts-1/wal/wals/9352bc48c30f4e3db10a8a4f45697319
I20260929 10:33:13.902128 18020 ts_tablet_manager.cc:1939] T 9352bc48c30f4e3db10a8a4f45697319 P 7f912be145e249078c19a61fb9f35c53: Deleting consensus metadata
I20260929 10:33:13.902263 17411 catalog_manager.cc:4841] TS 2b9cd0624fa84bc5b1a0de85bce66a72 (127.15.82.67:40743): tablet 9352bc48c30f4e3db10a8a4f45697319 (table test-table [id=c16ba1485a7c4285b5b4fcf594e1e00c]) successfully deleted
I20260929 10:33:13.902469 17409 catalog_manager.cc:4841] TS 1f1282b07b1f4d5c95e009991f21abbd (127.15.82.65:45387): tablet 9352bc48c30f4e3db10a8a4f45697319 (table test-table [id=c16ba1485a7c4285b5b4fcf594e1e00c]) successfully deleted
I20260929 10:33:13.902824 17409 catalog_manager.cc:4841] TS 7f912be145e249078c19a61fb9f35c53 (127.15.82.66:43273): tablet 9352bc48c30f4e3db10a8a4f45697319 (table test-table [id=c16ba1485a7c4285b5b4fcf594e1e00c]) successfully deleted
Sep 29 10:33:13 dist-test-slave-vg89 krb5kdc[17379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790677993, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 29 10:33:13 dist-test-slave-vg89 krb5kdc[17379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790677993, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:33:13.930536 18029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:13.917376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33114 (local address 127.15.82.126:39349)
0929 10:33:13.917673 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:13.917685 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:13.917706 (+ 21us) server_negotiation.cc:407] Connection header received
0929 10:33:13.917895 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:13.917899 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:13.917965 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:13.918061 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:13.918839 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:13.919415 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:13.920272 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:13.920514 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:13.920622 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:13.927423 (+ 6801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:13.927427 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:13.927429 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:13.927456 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:13.928788 (+ 1332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:13.929215 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:13.929219 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:13.929221 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:13.929308 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:13.929596 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:13.929600 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:13.929602 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:13.929798 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:13.929949 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:13.930181 (+ 232us) server_negotiation.cc:299] Negotiation successful
0929 10:33:13.930313 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":111,"threads_started":1}
I20260929 10:33:13.933058 17422 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33114:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 10:33:13.933511 17422 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:33:13.938200 17561 tablet_service.cc:1467] Processing CreateTablet for tablet db46fd3a49a84e9790bf13d6235d05ea (DEFAULT_TABLE table=test-table [id=3131ea0d4d534073bc79beda88600fa2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:33:13.938244 17701 tablet_service.cc:1467] Processing CreateTablet for tablet db46fd3a49a84e9790bf13d6235d05ea (DEFAULT_TABLE table=test-table [id=3131ea0d4d534073bc79beda88600fa2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:33:13.938542 17561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db46fd3a49a84e9790bf13d6235d05ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:33:13.938688 17841 tablet_service.cc:1467] Processing CreateTablet for tablet db46fd3a49a84e9790bf13d6235d05ea (DEFAULT_TABLE table=test-table [id=3131ea0d4d534073bc79beda88600fa2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:33:13.938990 17841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db46fd3a49a84e9790bf13d6235d05ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:33:13.940724 18030 tablet_bootstrap.cc:492] T db46fd3a49a84e9790bf13d6235d05ea P 1f1282b07b1f4d5c95e009991f21abbd: Bootstrap starting.
I20260929 10:33:13.941095 18031 tablet_bootstrap.cc:492] T db46fd3a49a84e9790bf13d6235d05ea P 2b9cd0624fa84bc5b1a0de85bce66a72: Bootstrap starting.
I20260929 10:33:13.941584 17701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db46fd3a49a84e9790bf13d6235d05ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:33:13.941673 18030 tablet_bootstrap.cc:654] T db46fd3a49a84e9790bf13d6235d05ea P 1f1282b07b1f4d5c95e009991f21abbd: Neither blocks nor log segments found. Creating new log.
I20260929 10:33:13.942035 18031 tablet_bootstrap.cc:654] T db46fd3a49a84e9790bf13d6235d05ea P 2b9cd0624fa84bc5b1a0de85bce66a72: Neither blocks nor log segments found. Creating new log.
I20260929 10:33:13.942883 18030 tablet_bootstrap.cc:492] T db46fd3a49a84e9790bf13d6235d05ea P 1f1282b07b1f4d5c95e009991f21abbd: No bootstrap required, opened a new log
I20260929 10:33:13.942994 18030 ts_tablet_manager.cc:1397] T db46fd3a49a84e9790bf13d6235d05ea P 1f1282b07b1f4d5c95e009991f21abbd: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 10:33:13.943212 18031 tablet_bootstrap.cc:492] T db46fd3a49a84e9790bf13d6235d05ea P 2b9cd0624fa84bc5b1a0de85bce66a72: No bootstrap required, opened a new log
I20260929 10:33:13.943298 18031 ts_tablet_manager.cc:1397] T db46fd3a49a84e9790bf13d6235d05ea P 2b9cd0624fa84bc5b1a0de85bce66a72: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 10:33:13.943454 18030 raft_consensus.cc:348] T db46fd3a49a84e9790bf13d6235d05ea P 1f1282b07b1f4d5c95e009991f21abbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1282b07b1f4d5c95e009991f21abbd" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45387 } } peers { permanent_uuid: "7f912be145e249078c19a61fb9f35c53" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43273 } } peers { permanent_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 40743 } }
I20260929 10:33:13.943564 18030 raft_consensus.cc:374] T db46fd3a49a84e9790bf13d6235d05ea P 1f1282b07b1f4d5c95e009991f21abbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:33:13.943598 18030 raft_consensus.cc:729] T db46fd3a49a84e9790bf13d6235d05ea P 1f1282b07b1f4d5c95e009991f21abbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f1282b07b1f4d5c95e009991f21abbd, State: Initialized, Role: FOLLOWER
I20260929 10:33:13.943745 18030 consensus_queue.cc:260] T db46fd3a49a84e9790bf13d6235d05ea P 1f1282b07b1f4d5c95e009991f21abbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1282b07b1f4d5c95e009991f21abbd" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45387 } } peers { permanent_uuid: "7f912be145e249078c19a61fb9f35c53" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43273 } } peers { permanent_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 40743 } }
I20260929 10:33:13.943776 18031 raft_consensus.cc:348] T db46fd3a49a84e9790bf13d6235d05ea P 2b9cd0624fa84bc5b1a0de85bce66a72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1282b07b1f4d5c95e009991f21abbd" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45387 } } peers { permanent_uuid: "7f912be145e249078c19a61fb9f35c53" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43273 } } peers { permanent_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 40743 } }
I20260929 10:33:13.943872 18031 raft_consensus.cc:374] T db46fd3a49a84e9790bf13d6235d05ea P 2b9cd0624fa84bc5b1a0de85bce66a72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:33:13.943912 18031 raft_consensus.cc:729] T db46fd3a49a84e9790bf13d6235d05ea P 2b9cd0624fa84bc5b1a0de85bce66a72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b9cd0624fa84bc5b1a0de85bce66a72, State: Initialized, Role: FOLLOWER
I20260929 10:33:13.944052 18031 consensus_queue.cc:260] T db46fd3a49a84e9790bf13d6235d05ea P 2b9cd0624fa84bc5b1a0de85bce66a72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1282b07b1f4d5c95e009991f21abbd" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45387 } } peers { permanent_uuid: "7f912be145e249078c19a61fb9f35c53" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43273 } } peers { permanent_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 40743 } }
I20260929 10:33:13.944306 18031 ts_tablet_manager.cc:1428] T db46fd3a49a84e9790bf13d6235d05ea P 2b9cd0624fa84bc5b1a0de85bce66a72: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260929 10:33:13.944770 18035 tablet_bootstrap.cc:492] T db46fd3a49a84e9790bf13d6235d05ea P 7f912be145e249078c19a61fb9f35c53: Bootstrap starting.
I20260929 10:33:13.944921 18030 ts_tablet_manager.cc:1428] T db46fd3a49a84e9790bf13d6235d05ea P 1f1282b07b1f4d5c95e009991f21abbd: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260929 10:33:13.945843 18035 tablet_bootstrap.cc:654] T db46fd3a49a84e9790bf13d6235d05ea P 7f912be145e249078c19a61fb9f35c53: Neither blocks nor log segments found. Creating new log.
I20260929 10:33:13.947726 18035 tablet_bootstrap.cc:492] T db46fd3a49a84e9790bf13d6235d05ea P 7f912be145e249078c19a61fb9f35c53: No bootstrap required, opened a new log
I20260929 10:33:13.947830 18035 ts_tablet_manager.cc:1397] T db46fd3a49a84e9790bf13d6235d05ea P 7f912be145e249078c19a61fb9f35c53: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260929 10:33:13.948194 18035 raft_consensus.cc:348] T db46fd3a49a84e9790bf13d6235d05ea P 7f912be145e249078c19a61fb9f35c53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1282b07b1f4d5c95e009991f21abbd" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45387 } } peers { permanent_uuid: "7f912be145e249078c19a61fb9f35c53" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43273 } } peers { permanent_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 40743 } }
I20260929 10:33:13.948309 18035 raft_consensus.cc:374] T db46fd3a49a84e9790bf13d6235d05ea P 7f912be145e249078c19a61fb9f35c53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:33:13.948354 18035 raft_consensus.cc:729] T db46fd3a49a84e9790bf13d6235d05ea P 7f912be145e249078c19a61fb9f35c53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f912be145e249078c19a61fb9f35c53, State: Initialized, Role: FOLLOWER
I20260929 10:33:13.948487 18035 consensus_queue.cc:260] T db46fd3a49a84e9790bf13d6235d05ea P 7f912be145e249078c19a61fb9f35c53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1282b07b1f4d5c95e009991f21abbd" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45387 } } peers { permanent_uuid: "7f912be145e249078c19a61fb9f35c53" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43273 } } peers { permanent_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 40743 } }
I20260929 10:33:13.948768 18035 ts_tablet_manager.cc:1428] T db46fd3a49a84e9790bf13d6235d05ea P 7f912be145e249078c19a61fb9f35c53: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260929 10:33:14.063997 18012 raft_consensus.cc:482] T db46fd3a49a84e9790bf13d6235d05ea P 7f912be145e249078c19a61fb9f35c53 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:33:14.064141 18012 raft_consensus.cc:504] T db46fd3a49a84e9790bf13d6235d05ea P 7f912be145e249078c19a61fb9f35c53 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1282b07b1f4d5c95e009991f21abbd" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45387 } } peers { permanent_uuid: "7f912be145e249078c19a61fb9f35c53" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43273 } } peers { permanent_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 40743 } }
I20260929 10:33:14.064581 18012 leader_election.cc:288] T db46fd3a49a84e9790bf13d6235d05ea P 7f912be145e249078c19a61fb9f35c53 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f1282b07b1f4d5c95e009991f21abbd (127.15.82.65:45387), 2b9cd0624fa84bc5b1a0de85bce66a72 (127.15.82.67:40743)
I20260929 10:33:14.065078 17580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db46fd3a49a84e9790bf13d6235d05ea" candidate_uuid: "7f912be145e249078c19a61fb9f35c53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f1282b07b1f4d5c95e009991f21abbd" is_pre_election: true
I20260929 10:33:14.065128 17861 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db46fd3a49a84e9790bf13d6235d05ea" candidate_uuid: "7f912be145e249078c19a61fb9f35c53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" is_pre_election: true
I20260929 10:33:14.065254 17580 raft_consensus.cc:2454] T db46fd3a49a84e9790bf13d6235d05ea P 1f1282b07b1f4d5c95e009991f21abbd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f912be145e249078c19a61fb9f35c53 in term 0.
I20260929 10:33:14.065303 17861 raft_consensus.cc:2454] T db46fd3a49a84e9790bf13d6235d05ea P 2b9cd0624fa84bc5b1a0de85bce66a72 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f912be145e249078c19a61fb9f35c53 in term 0.
I20260929 10:33:14.065654 17653 leader_election.cc:302] T db46fd3a49a84e9790bf13d6235d05ea P 7f912be145e249078c19a61fb9f35c53 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f1282b07b1f4d5c95e009991f21abbd, 7f912be145e249078c19a61fb9f35c53; no voters:
I20260929 10:33:14.065857 18012 raft_consensus.cc:2781] T db46fd3a49a84e9790bf13d6235d05ea P 7f912be145e249078c19a61fb9f35c53 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:33:14.065918 18012 raft_consensus.cc:482] T db46fd3a49a84e9790bf13d6235d05ea P 7f912be145e249078c19a61fb9f35c53 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:33:14.065960 18012 raft_consensus.cc:3037] T db46fd3a49a84e9790bf13d6235d05ea P 7f912be145e249078c19a61fb9f35c53 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:33:14.067025 18012 raft_consensus.cc:504] T db46fd3a49a84e9790bf13d6235d05ea P 7f912be145e249078c19a61fb9f35c53 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1282b07b1f4d5c95e009991f21abbd" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45387 } } peers { permanent_uuid: "7f912be145e249078c19a61fb9f35c53" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43273 } } peers { permanent_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 40743 } }
I20260929 10:33:14.067432 18012 leader_election.cc:288] T db46fd3a49a84e9790bf13d6235d05ea P 7f912be145e249078c19a61fb9f35c53 [CANDIDATE]: Term 1 election: Requested vote from peers 1f1282b07b1f4d5c95e009991f21abbd (127.15.82.65:45387), 2b9cd0624fa84bc5b1a0de85bce66a72 (127.15.82.67:40743)
I20260929 10:33:14.067804 17580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db46fd3a49a84e9790bf13d6235d05ea" candidate_uuid: "7f912be145e249078c19a61fb9f35c53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f1282b07b1f4d5c95e009991f21abbd"
I20260929 10:33:14.067804 17861 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db46fd3a49a84e9790bf13d6235d05ea" candidate_uuid: "7f912be145e249078c19a61fb9f35c53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72"
I20260929 10:33:14.067935 17861 raft_consensus.cc:3037] T db46fd3a49a84e9790bf13d6235d05ea P 2b9cd0624fa84bc5b1a0de85bce66a72 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:33:14.067945 17580 raft_consensus.cc:3037] T db46fd3a49a84e9790bf13d6235d05ea P 1f1282b07b1f4d5c95e009991f21abbd [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:33:14.069141 17580 raft_consensus.cc:2454] T db46fd3a49a84e9790bf13d6235d05ea P 1f1282b07b1f4d5c95e009991f21abbd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f912be145e249078c19a61fb9f35c53 in term 1.
I20260929 10:33:14.069141 17861 raft_consensus.cc:2454] T db46fd3a49a84e9790bf13d6235d05ea P 2b9cd0624fa84bc5b1a0de85bce66a72 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f912be145e249078c19a61fb9f35c53 in term 1.
I20260929 10:33:14.069434 17655 leader_election.cc:302] T db46fd3a49a84e9790bf13d6235d05ea P 7f912be145e249078c19a61fb9f35c53 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1f1282b07b1f4d5c95e009991f21abbd, 2b9cd0624fa84bc5b1a0de85bce66a72, 7f912be145e249078c19a61fb9f35c53; no voters:
I20260929 10:33:14.069641 18012 raft_consensus.cc:2781] T db46fd3a49a84e9790bf13d6235d05ea P 7f912be145e249078c19a61fb9f35c53 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:33:14.069713 18012 raft_consensus.cc:686] T db46fd3a49a84e9790bf13d6235d05ea P 7f912be145e249078c19a61fb9f35c53 [term 1 LEADER]: Becoming Leader. State: Replica: 7f912be145e249078c19a61fb9f35c53, State: Running, Role: LEADER
I20260929 10:33:14.069847 18012 consensus_queue.cc:237] T db46fd3a49a84e9790bf13d6235d05ea P 7f912be145e249078c19a61fb9f35c53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1282b07b1f4d5c95e009991f21abbd" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45387 } } peers { permanent_uuid: "7f912be145e249078c19a61fb9f35c53" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43273 } } peers { permanent_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 40743 } }
I20260929 10:33:14.071681 17422 catalog_manager.cc:5495] T db46fd3a49a84e9790bf13d6235d05ea P 7f912be145e249078c19a61fb9f35c53 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7f912be145e249078c19a61fb9f35c53 (127.15.82.66). New cstate: current_term: 1 leader_uuid: "7f912be145e249078c19a61fb9f35c53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1282b07b1f4d5c95e009991f21abbd" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f912be145e249078c19a61fb9f35c53" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43273 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 40743 } health_report { overall_health: UNKNOWN } } }
I20260929 10:33:14.102823 18039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:14.099836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53716 (local address 127.15.82.66:43273)
0929 10:33:14.100109 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:14.100116 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:14.100128 (+ 12us) server_negotiation.cc:407] Connection header received
0929 10:33:14.100165 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:14.100167 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:14.100204 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:14.100272 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:33:14.100505 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:14.101035 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:14.101837 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:14.102060 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:14.102132 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:14.102352 (+ 220us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:33:14.102439 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:33:14.102492 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:14.102588 (+ 96us) server_negotiation.cc:299] Negotiation successful
0929 10:33:14.102629 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":124,"threads_started":1}
I20260929 10:33:14.108610 17580 raft_consensus.cc:1261] T db46fd3a49a84e9790bf13d6235d05ea P 1f1282b07b1f4d5c95e009991f21abbd [term 1 FOLLOWER]: Refusing update from remote peer 7f912be145e249078c19a61fb9f35c53: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:33:14.108670 17861 raft_consensus.cc:1261] T db46fd3a49a84e9790bf13d6235d05ea P 2b9cd0624fa84bc5b1a0de85bce66a72 [term 1 FOLLOWER]: Refusing update from remote peer 7f912be145e249078c19a61fb9f35c53: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:33:14.109057 18013 consensus_queue.cc:1035] T db46fd3a49a84e9790bf13d6235d05ea P 7f912be145e249078c19a61fb9f35c53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 40743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:33:14.109205 18012 consensus_queue.cc:1035] T db46fd3a49a84e9790bf13d6235d05ea P 7f912be145e249078c19a61fb9f35c53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f1282b07b1f4d5c95e009991f21abbd" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:33:14.139055 17422 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33114:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:33:14.139233 17422 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33114:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:33:14.141227 17422 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3639d14359334e7fbfb18a16004331c8: Sending DeleteTablet for 3 replicas of tablet db46fd3a49a84e9790bf13d6235d05ea
I20260929 10:33:14.141789 17561 tablet_service.cc:1514] Processing DeleteTablet for tablet db46fd3a49a84e9790bf13d6235d05ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:33:14 UTC) from {username='kudu', principal='kudu/127.15.82.126@KRBTEST.COM'} at 127.0.0.1:44176
I20260929 10:33:14.141865 17841 tablet_service.cc:1514] Processing DeleteTablet for tablet db46fd3a49a84e9790bf13d6235d05ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:33:14 UTC) from {username='kudu', principal='kudu/127.15.82.126@KRBTEST.COM'} at 127.0.0.1:59772
I20260929 10:33:14.142009 17701 tablet_service.cc:1514] Processing DeleteTablet for tablet db46fd3a49a84e9790bf13d6235d05ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:33:14 UTC) from {username='kudu', principal='kudu/127.15.82.126@KRBTEST.COM'} at 127.0.0.1:53748
I20260929 10:33:14.142211 18020 tablet_replica.cc:330] T db46fd3a49a84e9790bf13d6235d05ea P 7f912be145e249078c19a61fb9f35c53: stopping tablet replica
I20260929 10:33:14.142211 18019 tablet_replica.cc:330] T db46fd3a49a84e9790bf13d6235d05ea P 1f1282b07b1f4d5c95e009991f21abbd: stopping tablet replica
I20260929 10:33:14.142346 18020 raft_consensus.cc:2229] T db46fd3a49a84e9790bf13d6235d05ea P 7f912be145e249078c19a61fb9f35c53 [term 1 LEADER]: Raft consensus shutting down.
I20260929 10:33:14.142345 18019 raft_consensus.cc:2229] T db46fd3a49a84e9790bf13d6235d05ea P 1f1282b07b1f4d5c95e009991f21abbd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:33:14.142390 18021 tablet_replica.cc:330] T db46fd3a49a84e9790bf13d6235d05ea P 2b9cd0624fa84bc5b1a0de85bce66a72: stopping tablet replica
I20260929 10:33:14.142503 18019 raft_consensus.cc:2258] T db46fd3a49a84e9790bf13d6235d05ea P 1f1282b07b1f4d5c95e009991f21abbd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:33:14.142536 18021 raft_consensus.cc:2229] T db46fd3a49a84e9790bf13d6235d05ea P 2b9cd0624fa84bc5b1a0de85bce66a72 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:33:14.142566 18020 raft_consensus.cc:2258] T db46fd3a49a84e9790bf13d6235d05ea P 7f912be145e249078c19a61fb9f35c53 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:33:14.142640 18021 raft_consensus.cc:2258] T db46fd3a49a84e9790bf13d6235d05ea P 2b9cd0624fa84bc5b1a0de85bce66a72 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:33:14.142730 17422 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33114:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 10:33:14.143256 17422 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:33:14.143298 18019 ts_tablet_manager.cc:1905] T db46fd3a49a84e9790bf13d6235d05ea P 1f1282b07b1f4d5c95e009991f21abbd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:33:14.143437 18021 ts_tablet_manager.cc:1905] T db46fd3a49a84e9790bf13d6235d05ea P 2b9cd0624fa84bc5b1a0de85bce66a72: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:33:14.143499 18020 ts_tablet_manager.cc:1905] T db46fd3a49a84e9790bf13d6235d05ea P 7f912be145e249078c19a61fb9f35c53: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:33:14.145818 18019 ts_tablet_manager.cc:1918] T db46fd3a49a84e9790bf13d6235d05ea P 1f1282b07b1f4d5c95e009991f21abbd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 10:33:14.145818 18021 ts_tablet_manager.cc:1918] T db46fd3a49a84e9790bf13d6235d05ea P 2b9cd0624fa84bc5b1a0de85bce66a72: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 10:33:14.145902 18021 log.cc:1196] T db46fd3a49a84e9790bf13d6235d05ea P 2b9cd0624fa84bc5b1a0de85bce66a72: Deleting WAL directory at /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/ts-2/wal/wals/db46fd3a49a84e9790bf13d6235d05ea
I20260929 10:33:14.145902 18019 log.cc:1196] T db46fd3a49a84e9790bf13d6235d05ea P 1f1282b07b1f4d5c95e009991f21abbd: Deleting WAL directory at /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/ts-0/wal/wals/db46fd3a49a84e9790bf13d6235d05ea
I20260929 10:33:14.146180 18021 ts_tablet_manager.cc:1939] T db46fd3a49a84e9790bf13d6235d05ea P 2b9cd0624fa84bc5b1a0de85bce66a72: Deleting consensus metadata
I20260929 10:33:14.146180 18019 ts_tablet_manager.cc:1939] T db46fd3a49a84e9790bf13d6235d05ea P 1f1282b07b1f4d5c95e009991f21abbd: Deleting consensus metadata
I20260929 10:33:14.146165 18020 ts_tablet_manager.cc:1918] T db46fd3a49a84e9790bf13d6235d05ea P 7f912be145e249078c19a61fb9f35c53: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 10:33:14.146265 18020 log.cc:1196] T db46fd3a49a84e9790bf13d6235d05ea P 7f912be145e249078c19a61fb9f35c53: Deleting WAL directory at /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/ts-1/wal/wals/db46fd3a49a84e9790bf13d6235d05ea
I20260929 10:33:14.146576 18020 ts_tablet_manager.cc:1939] T db46fd3a49a84e9790bf13d6235d05ea P 7f912be145e249078c19a61fb9f35c53: Deleting consensus metadata
I20260929 10:33:14.146872 17409 catalog_manager.cc:4841] TS 1f1282b07b1f4d5c95e009991f21abbd (127.15.82.65:45387): tablet db46fd3a49a84e9790bf13d6235d05ea (table test-table [id=3131ea0d4d534073bc79beda88600fa2]) successfully deleted
I20260929 10:33:14.147315 17409 catalog_manager.cc:4841] TS 7f912be145e249078c19a61fb9f35c53 (127.15.82.66:43273): tablet db46fd3a49a84e9790bf13d6235d05ea (table test-table [id=3131ea0d4d534073bc79beda88600fa2]) successfully deleted
I20260929 10:33:14.147977 17411 catalog_manager.cc:4841] TS 2b9cd0624fa84bc5b1a0de85bce66a72 (127.15.82.67:40743): tablet db46fd3a49a84e9790bf13d6235d05ea (table test-table [id=3131ea0d4d534073bc79beda88600fa2]) successfully deleted
I20260929 10:33:14.149080 17701 tablet_service.cc:1467] Processing CreateTablet for tablet 5bfd40dbb8204e008b083cb83463ec95 (DEFAULT_TABLE table=test-table [id=f58fdc3cad854ae187eb5925a441cf2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:33:14.149082 17561 tablet_service.cc:1467] Processing CreateTablet for tablet 5bfd40dbb8204e008b083cb83463ec95 (DEFAULT_TABLE table=test-table [id=f58fdc3cad854ae187eb5925a441cf2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:33:14.149205 17841 tablet_service.cc:1467] Processing CreateTablet for tablet 5bfd40dbb8204e008b083cb83463ec95 (DEFAULT_TABLE table=test-table [id=f58fdc3cad854ae187eb5925a441cf2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:33:14.149284 17701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5bfd40dbb8204e008b083cb83463ec95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:33:14.149323 17561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5bfd40dbb8204e008b083cb83463ec95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:33:14.149418 17841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5bfd40dbb8204e008b083cb83463ec95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:33:14.151254 18035 tablet_bootstrap.cc:492] T 5bfd40dbb8204e008b083cb83463ec95 P 7f912be145e249078c19a61fb9f35c53: Bootstrap starting.
I20260929 10:33:14.151552 18030 tablet_bootstrap.cc:492] T 5bfd40dbb8204e008b083cb83463ec95 P 1f1282b07b1f4d5c95e009991f21abbd: Bootstrap starting.
I20260929 10:33:14.152048 18035 tablet_bootstrap.cc:654] T 5bfd40dbb8204e008b083cb83463ec95 P 7f912be145e249078c19a61fb9f35c53: Neither blocks nor log segments found. Creating new log.
I20260929 10:33:14.152269 18030 tablet_bootstrap.cc:654] T 5bfd40dbb8204e008b083cb83463ec95 P 1f1282b07b1f4d5c95e009991f21abbd: Neither blocks nor log segments found. Creating new log.
I20260929 10:33:14.152792 18031 tablet_bootstrap.cc:492] T 5bfd40dbb8204e008b083cb83463ec95 P 2b9cd0624fa84bc5b1a0de85bce66a72: Bootstrap starting.
I20260929 10:33:14.153293 18030 tablet_bootstrap.cc:492] T 5bfd40dbb8204e008b083cb83463ec95 P 1f1282b07b1f4d5c95e009991f21abbd: No bootstrap required, opened a new log
I20260929 10:33:14.153350 18030 ts_tablet_manager.cc:1397] T 5bfd40dbb8204e008b083cb83463ec95 P 1f1282b07b1f4d5c95e009991f21abbd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 10:33:14.153551 18031 tablet_bootstrap.cc:654] T 5bfd40dbb8204e008b083cb83463ec95 P 2b9cd0624fa84bc5b1a0de85bce66a72: Neither blocks nor log segments found. Creating new log.
I20260929 10:33:14.153647 18030 raft_consensus.cc:348] T 5bfd40dbb8204e008b083cb83463ec95 P 1f1282b07b1f4d5c95e009991f21abbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1282b07b1f4d5c95e009991f21abbd" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45387 } } peers { permanent_uuid: "7f912be145e249078c19a61fb9f35c53" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43273 } } peers { permanent_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 40743 } }
I20260929 10:33:14.153728 18030 raft_consensus.cc:374] T 5bfd40dbb8204e008b083cb83463ec95 P 1f1282b07b1f4d5c95e009991f21abbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:33:14.153750 18030 raft_consensus.cc:729] T 5bfd40dbb8204e008b083cb83463ec95 P 1f1282b07b1f4d5c95e009991f21abbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f1282b07b1f4d5c95e009991f21abbd, State: Initialized, Role: FOLLOWER
I20260929 10:33:14.153884 18030 consensus_queue.cc:260] T 5bfd40dbb8204e008b083cb83463ec95 P 1f1282b07b1f4d5c95e009991f21abbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1282b07b1f4d5c95e009991f21abbd" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45387 } } peers { permanent_uuid: "7f912be145e249078c19a61fb9f35c53" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43273 } } peers { permanent_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 40743 } }
I20260929 10:33:14.154172 18030 ts_tablet_manager.cc:1428] T 5bfd40dbb8204e008b083cb83463ec95 P 1f1282b07b1f4d5c95e009991f21abbd: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260929 10:33:14.154805 18031 tablet_bootstrap.cc:492] T 5bfd40dbb8204e008b083cb83463ec95 P 2b9cd0624fa84bc5b1a0de85bce66a72: No bootstrap required, opened a new log
I20260929 10:33:14.154912 18031 ts_tablet_manager.cc:1397] T 5bfd40dbb8204e008b083cb83463ec95 P 2b9cd0624fa84bc5b1a0de85bce66a72: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 10:33:14.155293 18031 raft_consensus.cc:348] T 5bfd40dbb8204e008b083cb83463ec95 P 2b9cd0624fa84bc5b1a0de85bce66a72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1282b07b1f4d5c95e009991f21abbd" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45387 } } peers { permanent_uuid: "7f912be145e249078c19a61fb9f35c53" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43273 } } peers { permanent_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 40743 } }
I20260929 10:33:14.155375 18031 raft_consensus.cc:374] T 5bfd40dbb8204e008b083cb83463ec95 P 2b9cd0624fa84bc5b1a0de85bce66a72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:33:14.155395 18031 raft_consensus.cc:729] T 5bfd40dbb8204e008b083cb83463ec95 P 2b9cd0624fa84bc5b1a0de85bce66a72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b9cd0624fa84bc5b1a0de85bce66a72, State: Initialized, Role: FOLLOWER
I20260929 10:33:14.155529 18031 consensus_queue.cc:260] T 5bfd40dbb8204e008b083cb83463ec95 P 2b9cd0624fa84bc5b1a0de85bce66a72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1282b07b1f4d5c95e009991f21abbd" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45387 } } peers { permanent_uuid: "7f912be145e249078c19a61fb9f35c53" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43273 } } peers { permanent_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 40743 } }
I20260929 10:33:14.155818 18031 ts_tablet_manager.cc:1428] T 5bfd40dbb8204e008b083cb83463ec95 P 2b9cd0624fa84bc5b1a0de85bce66a72: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260929 10:33:14.156322 18035 tablet_bootstrap.cc:492] T 5bfd40dbb8204e008b083cb83463ec95 P 7f912be145e249078c19a61fb9f35c53: No bootstrap required, opened a new log
I20260929 10:33:14.156419 18035 ts_tablet_manager.cc:1397] T 5bfd40dbb8204e008b083cb83463ec95 P 7f912be145e249078c19a61fb9f35c53: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260929 10:33:14.156795 18035 raft_consensus.cc:348] T 5bfd40dbb8204e008b083cb83463ec95 P 7f912be145e249078c19a61fb9f35c53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1282b07b1f4d5c95e009991f21abbd" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45387 } } peers { permanent_uuid: "7f912be145e249078c19a61fb9f35c53" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43273 } } peers { permanent_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 40743 } }
I20260929 10:33:14.156881 18035 raft_consensus.cc:374] T 5bfd40dbb8204e008b083cb83463ec95 P 7f912be145e249078c19a61fb9f35c53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:33:14.156903 18035 raft_consensus.cc:729] T 5bfd40dbb8204e008b083cb83463ec95 P 7f912be145e249078c19a61fb9f35c53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f912be145e249078c19a61fb9f35c53, State: Initialized, Role: FOLLOWER
I20260929 10:33:14.157043 18035 consensus_queue.cc:260] T 5bfd40dbb8204e008b083cb83463ec95 P 7f912be145e249078c19a61fb9f35c53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1282b07b1f4d5c95e009991f21abbd" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45387 } } peers { permanent_uuid: "7f912be145e249078c19a61fb9f35c53" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43273 } } peers { permanent_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 40743 } }
I20260929 10:33:14.157325 18035 ts_tablet_manager.cc:1428] T 5bfd40dbb8204e008b083cb83463ec95 P 7f912be145e249078c19a61fb9f35c53: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260929 10:33:14.188696 18005 raft_consensus.cc:482] T 5bfd40dbb8204e008b083cb83463ec95 P 2b9cd0624fa84bc5b1a0de85bce66a72 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:33:14.188838 18005 raft_consensus.cc:504] T 5bfd40dbb8204e008b083cb83463ec95 P 2b9cd0624fa84bc5b1a0de85bce66a72 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1282b07b1f4d5c95e009991f21abbd" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45387 } } peers { permanent_uuid: "7f912be145e249078c19a61fb9f35c53" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43273 } } peers { permanent_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 40743 } }
I20260929 10:33:14.189196 18005 leader_election.cc:288] T 5bfd40dbb8204e008b083cb83463ec95 P 2b9cd0624fa84bc5b1a0de85bce66a72 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f1282b07b1f4d5c95e009991f21abbd (127.15.82.65:45387), 7f912be145e249078c19a61fb9f35c53 (127.15.82.66:43273)
I20260929 10:33:14.189750 17721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5bfd40dbb8204e008b083cb83463ec95" candidate_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f912be145e249078c19a61fb9f35c53" is_pre_election: true
I20260929 10:33:14.189742 17580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5bfd40dbb8204e008b083cb83463ec95" candidate_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f1282b07b1f4d5c95e009991f21abbd" is_pre_election: true
I20260929 10:33:14.189929 17721 raft_consensus.cc:2454] T 5bfd40dbb8204e008b083cb83463ec95 P 7f912be145e249078c19a61fb9f35c53 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b9cd0624fa84bc5b1a0de85bce66a72 in term 0.
I20260929 10:33:14.189926 17580 raft_consensus.cc:2454] T 5bfd40dbb8204e008b083cb83463ec95 P 1f1282b07b1f4d5c95e009991f21abbd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b9cd0624fa84bc5b1a0de85bce66a72 in term 0.
I20260929 10:33:14.190306 17793 leader_election.cc:302] T 5bfd40dbb8204e008b083cb83463ec95 P 2b9cd0624fa84bc5b1a0de85bce66a72 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f1282b07b1f4d5c95e009991f21abbd, 2b9cd0624fa84bc5b1a0de85bce66a72; no voters:
I20260929 10:33:14.190536 18005 raft_consensus.cc:2781] T 5bfd40dbb8204e008b083cb83463ec95 P 2b9cd0624fa84bc5b1a0de85bce66a72 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:33:14.190601 18005 raft_consensus.cc:482] T 5bfd40dbb8204e008b083cb83463ec95 P 2b9cd0624fa84bc5b1a0de85bce66a72 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:33:14.190629 18005 raft_consensus.cc:3037] T 5bfd40dbb8204e008b083cb83463ec95 P 2b9cd0624fa84bc5b1a0de85bce66a72 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:33:14.191697 18005 raft_consensus.cc:504] T 5bfd40dbb8204e008b083cb83463ec95 P 2b9cd0624fa84bc5b1a0de85bce66a72 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1282b07b1f4d5c95e009991f21abbd" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45387 } } peers { permanent_uuid: "7f912be145e249078c19a61fb9f35c53" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43273 } } peers { permanent_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 40743 } }
I20260929 10:33:14.192021 18005 leader_election.cc:288] T 5bfd40dbb8204e008b083cb83463ec95 P 2b9cd0624fa84bc5b1a0de85bce66a72 [CANDIDATE]: Term 1 election: Requested vote from peers 1f1282b07b1f4d5c95e009991f21abbd (127.15.82.65:45387), 7f912be145e249078c19a61fb9f35c53 (127.15.82.66:43273)
I20260929 10:33:14.192458 17580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5bfd40dbb8204e008b083cb83463ec95" candidate_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f1282b07b1f4d5c95e009991f21abbd"
I20260929 10:33:14.192495 17721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5bfd40dbb8204e008b083cb83463ec95" candidate_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f912be145e249078c19a61fb9f35c53"
I20260929 10:33:14.192567 17580 raft_consensus.cc:3037] T 5bfd40dbb8204e008b083cb83463ec95 P 1f1282b07b1f4d5c95e009991f21abbd [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:33:14.192636 17721 raft_consensus.cc:3037] T 5bfd40dbb8204e008b083cb83463ec95 P 7f912be145e249078c19a61fb9f35c53 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:33:14.193811 17580 raft_consensus.cc:2454] T 5bfd40dbb8204e008b083cb83463ec95 P 1f1282b07b1f4d5c95e009991f21abbd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b9cd0624fa84bc5b1a0de85bce66a72 in term 1.
I20260929 10:33:14.193970 17721 raft_consensus.cc:2454] T 5bfd40dbb8204e008b083cb83463ec95 P 7f912be145e249078c19a61fb9f35c53 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b9cd0624fa84bc5b1a0de85bce66a72 in term 1.
I20260929 10:33:14.194108 17793 leader_election.cc:302] T 5bfd40dbb8204e008b083cb83463ec95 P 2b9cd0624fa84bc5b1a0de85bce66a72 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f1282b07b1f4d5c95e009991f21abbd, 2b9cd0624fa84bc5b1a0de85bce66a72; no voters:
I20260929 10:33:14.194295 18005 raft_consensus.cc:2781] T 5bfd40dbb8204e008b083cb83463ec95 P 2b9cd0624fa84bc5b1a0de85bce66a72 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:33:14.194389 18005 raft_consensus.cc:686] T 5bfd40dbb8204e008b083cb83463ec95 P 2b9cd0624fa84bc5b1a0de85bce66a72 [term 1 LEADER]: Becoming Leader. State: Replica: 2b9cd0624fa84bc5b1a0de85bce66a72, State: Running, Role: LEADER
I20260929 10:33:14.194581 18005 consensus_queue.cc:237] T 5bfd40dbb8204e008b083cb83463ec95 P 2b9cd0624fa84bc5b1a0de85bce66a72 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1282b07b1f4d5c95e009991f21abbd" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45387 } } peers { permanent_uuid: "7f912be145e249078c19a61fb9f35c53" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43273 } } peers { permanent_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 40743 } }
I20260929 10:33:14.196260 17422 catalog_manager.cc:5495] T 5bfd40dbb8204e008b083cb83463ec95 P 2b9cd0624fa84bc5b1a0de85bce66a72 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b9cd0624fa84bc5b1a0de85bce66a72 (127.15.82.67). New cstate: current_term: 1 leader_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1282b07b1f4d5c95e009991f21abbd" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f912be145e249078c19a61fb9f35c53" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b9cd0624fa84bc5b1a0de85bce66a72" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 40743 } health_report { overall_health: HEALTHY } } }
I20260929 10:33:14.216351 18048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:14.213114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33082 (local address 127.15.82.67:40743)
0929 10:33:14.213397 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:14.213406 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:14.213418 (+ 12us) server_negotiation.cc:407] Connection header received
0929 10:33:14.213460 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:14.213462 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:14.213501 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:14.213597 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:33:14.213850 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:14.214431 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:14.215277 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:14.215515 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:14.215617 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:14.215897 (+ 280us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:33:14.215968 (+ 71us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:33:14.216049 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:14.216138 (+ 89us) server_negotiation.cc:299] Negotiation successful
0929 10:33:14.216188 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":93,"threads_started":1}
I20260929 10:33:14.224951 18051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:14.221618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.67:40743 (local address 127.15.82.67:46457)
0929 10:33:14.221897 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:14.221913 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:14.222039 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:14.222261 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:14.222263 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:14.222269 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 10:33:14.222465 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:14.222472 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:14.223168 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:14.223174 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:14.223766 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:14.223773 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:14.224265 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:14.224269 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:14.224354 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:14.224382 (+ 28us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 10:33:14.224640 (+ 258us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 10:33:14.224647 (+ 7us) client_negotiation.cc:769] Sending connection context
0929 10:33:14.224712 (+ 65us) client_negotiation.cc:240] Negotiation successful
0929 10:33:14.224784 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"mutex_wait_us":18,"thread_start_us":115,"threads_started":1}
I20260929 10:33:14.224951 18048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:14.221784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.67:46457 (local address 127.15.82.67:40743)
0929 10:33:14.221894 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:14.221898 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:14.221995 (+ 97us) server_negotiation.cc:407] Connection header received
0929 10:33:14.222131 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:14.222134 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:14.222172 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:14.222254 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:33:14.222569 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:14.223068 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:14.223880 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:14.224162 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:14.224266 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:14.224490 (+ 224us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:33:14.224561 (+ 71us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:33:14.224639 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:14.224732 (+ 93us) server_negotiation.cc:299] Negotiation successful
0929 10:33:14.224772 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":88,"server-negotiator.queue_time_us":57}
I20260929 10:33:14.230376 17580 raft_consensus.cc:1261] T 5bfd40dbb8204e008b083cb83463ec95 P 1f1282b07b1f4d5c95e009991f21abbd [term 1 FOLLOWER]: Refusing update from remote peer 2b9cd0624fa84bc5b1a0de85bce66a72: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:33:14.230756 17721 raft_consensus.cc:1261] T 5bfd40dbb8204e008b083cb83463ec95 P 7f912be145e249078c19a61fb9f35c53 [term 1 FOLLOWER]: Refusing update from remote peer 2b9cd0624fa84bc5b1a0de85bce66a72: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:33:14.230818 17997 consensus_queue.cc:1035] T 5bfd40dbb8204e008b083cb83463ec95 P 2b9cd0624fa84bc5b1a0de85bce66a72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f1282b07b1f4d5c95e009991f21abbd" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:33:14.231144 17997 consensus_queue.cc:1035] T 5bfd40dbb8204e008b083cb83463ec95 P 2b9cd0624fa84bc5b1a0de85bce66a72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f912be145e249078c19a61fb9f35c53" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260929 10:33:14.244473 17998 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260929 10:33:15.800860 17423 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33114:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:33:15.801007 17423 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33114:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:33:15.803356 17423 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3639d14359334e7fbfb18a16004331c8: Sending DeleteTablet for 3 replicas of tablet 5bfd40dbb8204e008b083cb83463ec95
I20260929 10:33:15.803936 17841 tablet_service.cc:1514] Processing DeleteTablet for tablet 5bfd40dbb8204e008b083cb83463ec95 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:33:15 UTC) from {username='kudu', principal='kudu/127.15.82.126@KRBTEST.COM'} at 127.0.0.1:59772
I20260929 10:33:15.803964 17561 tablet_service.cc:1514] Processing DeleteTablet for tablet 5bfd40dbb8204e008b083cb83463ec95 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:33:15 UTC) from {username='kudu', principal='kudu/127.15.82.126@KRBTEST.COM'} at 127.0.0.1:44176
I20260929 10:33:15.804309 18076 tablet_replica.cc:330] T 5bfd40dbb8204e008b083cb83463ec95 P 1f1282b07b1f4d5c95e009991f21abbd: stopping tablet replica
I20260929 10:33:15.804355 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 17496
I20260929 10:33:15.804941 17701 tablet_service.cc:1514] Processing DeleteTablet for tablet 5bfd40dbb8204e008b083cb83463ec95 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:33:15 UTC) from {username='kudu', principal='kudu/127.15.82.126@KRBTEST.COM'} at 127.0.0.1:53748
I20260929 10:33:15.805832 18075 tablet_replica.cc:330] T 5bfd40dbb8204e008b083cb83463ec95 P 2b9cd0624fa84bc5b1a0de85bce66a72: stopping tablet replica
I20260929 10:33:15.805984 18075 raft_consensus.cc:2229] T 5bfd40dbb8204e008b083cb83463ec95 P 2b9cd0624fa84bc5b1a0de85bce66a72 [term 1 LEADER]: Raft consensus shutting down.
I20260929 10:33:15.806288 18075 raft_consensus.cc:2258] T 5bfd40dbb8204e008b083cb83463ec95 P 2b9cd0624fa84bc5b1a0de85bce66a72 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:33:15.806460 18077 tablet_replica.cc:330] T 5bfd40dbb8204e008b083cb83463ec95 P 7f912be145e249078c19a61fb9f35c53: stopping tablet replica
I20260929 10:33:15.806618 18077 raft_consensus.cc:2229] T 5bfd40dbb8204e008b083cb83463ec95 P 7f912be145e249078c19a61fb9f35c53 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:33:15.806740 18077 raft_consensus.cc:2258] T 5bfd40dbb8204e008b083cb83463ec95 P 7f912be145e249078c19a61fb9f35c53 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:33:15.807013 18075 ts_tablet_manager.cc:1905] T 5bfd40dbb8204e008b083cb83463ec95 P 2b9cd0624fa84bc5b1a0de85bce66a72: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:33:15.807438 18077 ts_tablet_manager.cc:1905] T 5bfd40dbb8204e008b083cb83463ec95 P 7f912be145e249078c19a61fb9f35c53: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:33:15.809221 18075 ts_tablet_manager.cc:1918] T 5bfd40dbb8204e008b083cb83463ec95 P 2b9cd0624fa84bc5b1a0de85bce66a72: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260929 10:33:15.809315 18075 log.cc:1196] T 5bfd40dbb8204e008b083cb83463ec95 P 2b9cd0624fa84bc5b1a0de85bce66a72: Deleting WAL directory at /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/ts-2/wal/wals/5bfd40dbb8204e008b083cb83463ec95
I20260929 10:33:15.809656 18075 ts_tablet_manager.cc:1939] T 5bfd40dbb8204e008b083cb83463ec95 P 2b9cd0624fa84bc5b1a0de85bce66a72: Deleting consensus metadata
I20260929 10:33:15.810436 17411 catalog_manager.cc:4841] TS 2b9cd0624fa84bc5b1a0de85bce66a72 (127.15.82.67:40743): tablet 5bfd40dbb8204e008b083cb83463ec95 (table test-table [id=f58fdc3cad854ae187eb5925a441cf2a]) successfully deleted
I20260929 10:33:15.810981 18077 ts_tablet_manager.cc:1918] T 5bfd40dbb8204e008b083cb83463ec95 P 7f912be145e249078c19a61fb9f35c53: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260929 10:33:15.811065 18077 log.cc:1196] T 5bfd40dbb8204e008b083cb83463ec95 P 7f912be145e249078c19a61fb9f35c53: Deleting WAL directory at /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790677981592896-15689-0/minicluster-data/ts-1/wal/wals/5bfd40dbb8204e008b083cb83463ec95
I20260929 10:33:15.811338 18077 ts_tablet_manager.cc:1939] T 5bfd40dbb8204e008b083cb83463ec95 P 7f912be145e249078c19a61fb9f35c53: Deleting consensus metadata
I20260929 10:33:15.812068 17409 catalog_manager.cc:4841] TS 7f912be145e249078c19a61fb9f35c53 (127.15.82.66:43273): tablet 5bfd40dbb8204e008b083cb83463ec95 (table test-table [id=f58fdc3cad854ae187eb5925a441cf2a]) successfully deleted
W20260929 10:33:15.813603 17409 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.82.65:45387 (error 108)
I20260929 10:33:15.813835 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 17636
I20260929 10:33:15.816447 18079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:15.815866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.65:45387 (local address 127.0.0.1:42862)
0929 10:33:15.816172 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:15.816284 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.65:45387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":103,"threads_started":1}
W20260929 10:33:15.816606 17409 catalog_manager.cc:4584] TS 1f1282b07b1f4d5c95e009991f21abbd (127.15.82.65:45387): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5bfd40dbb8204e008b083cb83463ec95: Network error: Client connection negotiation failed: client connection to 127.15.82.65:45387: connect: Connection refused (error 111)
I20260929 10:33:15.820776 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 17776
I20260929 10:33:15.827672 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 17392
2026-09-29T10:33:15Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6440 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:33:15 dist-test-slave-vg89 krb5kdc[18082](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:33:15 dist-test-slave-vg89 krb5kdc[18082](info): set up 2 sockets
Sep 29 10:33:15 dist-test-slave-vg89 krb5kdc[18082](info): commencing operation
krb5kdc: starting...
W20260929 10:33:16.882727 15689 mini_kdc.cc:121] Time spent starting KDC: real 1.031s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Sep 29 10:33:16 dist-test-slave-vg89 krb5kdc[18082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790677996, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:33:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:33:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.82.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab.
Entry for principal kudu/127.15.82.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab.
I20260929 10:33:17.004803 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:43605
--webserver_interface=127.15.82.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab
--principal=kudu/127.15.82.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:36819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.82.126:43605
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:33:17.114444 18095 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:33:17.114753 18095 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:33:17.114821 18095 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:33:17.118877 18095 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:33:17.118958 18095 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:33:17.118995 18095 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:33:17.119053 18095 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:33:17.119091 18095 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:33:17.124338 18095 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:36819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.82.126:43605
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab
--principal=kudu/127.15.82.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:43605
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.82.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18095
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:33:17.125422 18095 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:33:17.126510 18095 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:33:17.135264 18100 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:17.135320 18103 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:17.135285 18101 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:17.135305 18095 server_base.cc:1034] running on GCE node
I20260929 10:33:17.136111 18095 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:33:17.137214 18095 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:33:17.138443 18095 hybrid_clock.cc:648] HybridClock initialized: now 1790677997138436 us; error 40 us; skew 500 ppm
Sep 29 10:33:17 dist-test-slave-vg89 krb5kdc[18082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790677997, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:33:17.141294 18095 init.cc:377] Logged in from keytab as kudu/127.15.82.126@KRBTEST.COM (short username kudu)
I20260929 10:33:17.142129 18095 webserver.cc:460] Webserver started at http://127.15.82.126:35439/ using document root <none> and password file <none>
I20260929 10:33:17.142720 18095 fs_manager.cc:359] Metadata directory not provided
I20260929 10:33:17.142784 18095 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:33:17.142947 18095 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:33:17.144928 18095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/minicluster-data/master-0/data/instance:
uuid: "d7e5ccbd1ce14eaa95c46a39b1f4b49f"
format_stamp: "Formatted at 2026-09-29 10:33:17 on dist-test-slave-vg89"
server_key: "fb573d3ebab54eb8fbc967cf888e84e9"
server_key_iv: "814b772b3ecfce816fd7f1417fadddb4"
server_key_version: "encryptionkey@0"
I20260929 10:33:17.145556 18095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/minicluster-data/master-0/wal/instance:
uuid: "d7e5ccbd1ce14eaa95c46a39b1f4b49f"
format_stamp: "Formatted at 2026-09-29 10:33:17 on dist-test-slave-vg89"
server_key: "fb573d3ebab54eb8fbc967cf888e84e9"
server_key_iv: "814b772b3ecfce816fd7f1417fadddb4"
server_key_version: "encryptionkey@0"
I20260929 10:33:17.149389 18095 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260929 10:33:17.152038 18110 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:17.153270 18095 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260929 10:33:17.153453 18095 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/minicluster-data/master-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/minicluster-data/master-0/wal
uuid: "d7e5ccbd1ce14eaa95c46a39b1f4b49f"
format_stamp: "Formatted at 2026-09-29 10:33:17 on dist-test-slave-vg89"
server_key: "fb573d3ebab54eb8fbc967cf888e84e9"
server_key_iv: "814b772b3ecfce816fd7f1417fadddb4"
server_key_version: "encryptionkey@0"
I20260929 10:33:17.153577 18095 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:33:17.180107 18095 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:33:17.183426 18095 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:33:17.183642 18095 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:33:17.190626 18095 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.126:43605
I20260929 10:33:17.190654 18162 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.126:43605 every 8 connection(s)
I20260929 10:33:17.191593 18095 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
I20260929 10:33:17.194247 18163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:33:17.199579 18163 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7e5ccbd1ce14eaa95c46a39b1f4b49f: Bootstrap starting.
I20260929 10:33:17.201298 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 18095
I20260929 10:33:17.201418 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/minicluster-data/master-0/wal/instance
I20260929 10:33:17.201694 15689 external_mini_cluster.cc:1428] Setting key d17d1714909f6492d1e34de5a2a4aec3
I20260929 10:33:17.202353 18163 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d7e5ccbd1ce14eaa95c46a39b1f4b49f: Neither blocks nor log segments found. Creating new log.
I20260929 10:33:17.203296 18163 log.cc:824] T 00000000000000000000000000000000 P d7e5ccbd1ce14eaa95c46a39b1f4b49f: Log is configured to *not* fsync() on all Append() calls
I20260929 10:33:17.205084 18163 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7e5ccbd1ce14eaa95c46a39b1f4b49f: No bootstrap required, opened a new log
I20260929 10:33:17.208516 18163 raft_consensus.cc:348] T 00000000000000000000000000000000 P d7e5ccbd1ce14eaa95c46a39b1f4b49f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7e5ccbd1ce14eaa95c46a39b1f4b49f" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 43605 } }
I20260929 10:33:17.208793 18163 raft_consensus.cc:374] T 00000000000000000000000000000000 P d7e5ccbd1ce14eaa95c46a39b1f4b49f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:33:17.208889 18163 raft_consensus.cc:729] T 00000000000000000000000000000000 P d7e5ccbd1ce14eaa95c46a39b1f4b49f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e5ccbd1ce14eaa95c46a39b1f4b49f, State: Initialized, Role: FOLLOWER
I20260929 10:33:17.209383 18163 consensus_queue.cc:260] T 00000000000000000000000000000000 P d7e5ccbd1ce14eaa95c46a39b1f4b49f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7e5ccbd1ce14eaa95c46a39b1f4b49f" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 43605 } }
I20260929 10:33:17.209535 18163 raft_consensus.cc:388] T 00000000000000000000000000000000 P d7e5ccbd1ce14eaa95c46a39b1f4b49f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:33:17.209620 18163 raft_consensus.cc:482] T 00000000000000000000000000000000 P d7e5ccbd1ce14eaa95c46a39b1f4b49f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:33:17.209736 18163 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d7e5ccbd1ce14eaa95c46a39b1f4b49f [term 0 FOLLOWER]: Advancing to term 1
Sep 29 10:33:17 dist-test-slave-vg89 krb5kdc[18082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790677996, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:33:17.210914 18163 raft_consensus.cc:504] T 00000000000000000000000000000000 P d7e5ccbd1ce14eaa95c46a39b1f4b49f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7e5ccbd1ce14eaa95c46a39b1f4b49f" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 43605 } }
I20260929 10:33:17.211269 18163 leader_election.cc:302] T 00000000000000000000000000000000 P d7e5ccbd1ce14eaa95c46a39b1f4b49f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d7e5ccbd1ce14eaa95c46a39b1f4b49f; no voters:
I20260929 10:33:17.211586 18163 leader_election.cc:288] T 00000000000000000000000000000000 P d7e5ccbd1ce14eaa95c46a39b1f4b49f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:33:17.211704 18168 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d7e5ccbd1ce14eaa95c46a39b1f4b49f [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:33:17.211956 18168 raft_consensus.cc:686] T 00000000000000000000000000000000 P d7e5ccbd1ce14eaa95c46a39b1f4b49f [term 1 LEADER]: Becoming Leader. State: Replica: d7e5ccbd1ce14eaa95c46a39b1f4b49f, State: Running, Role: LEADER
I20260929 10:33:17.212284 18168 consensus_queue.cc:237] T 00000000000000000000000000000000 P d7e5ccbd1ce14eaa95c46a39b1f4b49f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7e5ccbd1ce14eaa95c46a39b1f4b49f" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 43605 } }
I20260929 10:33:17.212829 18163 sys_catalog.cc:564] T 00000000000000000000000000000000 P d7e5ccbd1ce14eaa95c46a39b1f4b49f [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:33:17.214133 18169 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7e5ccbd1ce14eaa95c46a39b1f4b49f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d7e5ccbd1ce14eaa95c46a39b1f4b49f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7e5ccbd1ce14eaa95c46a39b1f4b49f" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 43605 } } }
I20260929 10:33:17.214159 18170 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7e5ccbd1ce14eaa95c46a39b1f4b49f [sys.catalog]: SysCatalogTable state changed. Reason: New leader d7e5ccbd1ce14eaa95c46a39b1f4b49f. Latest consensus state: current_term: 1 leader_uuid: "d7e5ccbd1ce14eaa95c46a39b1f4b49f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7e5ccbd1ce14eaa95c46a39b1f4b49f" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 43605 } } }
I20260929 10:33:17.214253 18169 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7e5ccbd1ce14eaa95c46a39b1f4b49f [sys.catalog]: This master's current role is: LEADER
I20260929 10:33:17.214268 18170 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7e5ccbd1ce14eaa95c46a39b1f4b49f [sys.catalog]: This master's current role is: LEADER
I20260929 10:33:17.217900 18166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:17.202907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57790 (local address 127.15.82.126:43605)
0929 10:33:17.203356 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:17.203371 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:17.203409 (+ 38us) server_negotiation.cc:407] Connection header received
0929 10:33:17.204104 (+ 695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:17.204121 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:17.204443 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:17.204786 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:17.205634 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:17.206684 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:17.207567 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:17.208049 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:17.208229 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:17.211181 (+ 2952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:17.211186 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:17.211199 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:17.211226 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:17.213432 (+ 2206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:17.214827 (+ 1395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:17.214834 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:17.214841 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:17.214983 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:17.215377 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:17.215384 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:17.215389 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:17.215924 (+ 535us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:17.216119 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:17.216954 (+ 835us) server_negotiation.cc:299] Negotiation successful
0929 10:33:17.217191 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":135,"threads_started":1}
W20260929 10:33:17.218267 18183 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d7e5ccbd1ce14eaa95c46a39b1f4b49f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 10:33:17.218345 18183 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 10:33:17.218436 18177 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:33:17.219297 18177 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:33:17.224556 18177 catalog_manager.cc:1347] Generated new cluster ID: 49913a6953714eeb84b1b914c047072c
I20260929 10:33:17.224624 18177 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:33:17.233163 18177 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:33:17.234090 18177 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:33:17.244747 18177 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d7e5ccbd1ce14eaa95c46a39b1f4b49f: Generated new TSK 0
I20260929 10:33:17.245702 18177 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.82.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab.
Entry for principal kudu/127.15.82.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab.
I20260929 10:33:17.283699 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.65:0
--local_ip_for_outbound_sockets=127.15.82.65
--webserver_interface=127.15.82.65
--webserver_port=0
--tserver_master_addrs=127.15.82.126:43605
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab
--principal=kudu/127.15.82.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:36819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:33:17.417418 18189 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:33:17.417686 18189 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:33:17.417728 18189 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:33:17.421819 18189 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:33:17.421877 18189 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:33:17.421938 18189 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.65
I20260929 10:33:17.427083 18189 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:36819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab
--principal=kudu/127.15.82.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.82.65
--webserver_port=0
--tserver_master_addrs=127.15.82.126:43605
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18189
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.65
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:33:17.428087 18189 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:33:17.429083 18189 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:33:17.439258 18197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:17.439297 18195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:17.439409 18189 server_base.cc:1034] running on GCE node
W20260929 10:33:17.439680 18194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:17.440155 18189 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:33:17.440861 18189 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:33:17.442045 18189 hybrid_clock.cc:648] HybridClock initialized: now 1790677997442012 us; error 58 us; skew 500 ppm
Sep 29 10:33:17 dist-test-slave-vg89 krb5kdc[18082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790677997, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:33:17.445009 18189 init.cc:377] Logged in from keytab as kudu/127.15.82.65@KRBTEST.COM (short username kudu)
I20260929 10:33:17.445883 18189 webserver.cc:460] Webserver started at http://127.15.82.65:37817/ using document root <none> and password file <none>
I20260929 10:33:17.446491 18189 fs_manager.cc:359] Metadata directory not provided
I20260929 10:33:17.446558 18189 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:33:17.446728 18189 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:33:17.448710 18189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-0/data/instance:
uuid: "e08abaccb3dd40e7b5360cf881e149e5"
format_stamp: "Formatted at 2026-09-29 10:33:17 on dist-test-slave-vg89"
server_key: "ec28968843cffe3065a15e2c8498975b"
server_key_iv: "78e291286fc5b33c003cebef69c60f3b"
server_key_version: "encryptionkey@0"
I20260929 10:33:17.449208 18189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-0/wal/instance:
uuid: "e08abaccb3dd40e7b5360cf881e149e5"
format_stamp: "Formatted at 2026-09-29 10:33:17 on dist-test-slave-vg89"
server_key: "ec28968843cffe3065a15e2c8498975b"
server_key_iv: "78e291286fc5b33c003cebef69c60f3b"
server_key_version: "encryptionkey@0"
I20260929 10:33:17.452919 18189 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260929 10:33:17.455468 18204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:17.456835 18189 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 10:33:17.456955 18189 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-0/wal
uuid: "e08abaccb3dd40e7b5360cf881e149e5"
format_stamp: "Formatted at 2026-09-29 10:33:17 on dist-test-slave-vg89"
server_key: "ec28968843cffe3065a15e2c8498975b"
server_key_iv: "78e291286fc5b33c003cebef69c60f3b"
server_key_version: "encryptionkey@0"
I20260929 10:33:17.457067 18189 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:33:17.492815 18189 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:33:17.496232 18189 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:33:17.496451 18189 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:33:17.497072 18189 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:33:17.497920 18189 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:33:17.497967 18189 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:17.498031 18189 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:33:17.498061 18189 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:17.507057 18189 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.65:42285
I20260929 10:33:17.507082 18317 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.65:42285 every 8 connection(s)
I20260929 10:33:17.507975 18189 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
I20260929 10:33:17.511541 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 18189
I20260929 10:33:17.511633 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-0/wal/instance
I20260929 10:33:17.511933 15689 external_mini_cluster.cc:1428] Setting key c602bca269e5d41a4f8b7406aeb2bd71
Sep 29 10:33:17 dist-test-slave-vg89 krb5kdc[18082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790677997, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.65@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.82.66@KRBTEST.COM; defaulting to no policy
I20260929 10:33:17.522073 18166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:17.509817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.65:43759 (local address 127.15.82.126:43605)
0929 10:33:17.509955 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:17.509959 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:17.510789 (+ 830us) server_negotiation.cc:407] Connection header received
0929 10:33:17.512181 (+ 1392us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:17.512184 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:17.512243 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:17.512317 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:17.513712 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:17.514296 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:17.515571 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:17.515815 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:17.515922 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:17.518423 (+ 2501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:17.518429 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:17.518432 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:17.518462 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:17.519918 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:17.520461 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:17.520465 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:17.520467 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:17.520550 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:17.520891 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:17.520896 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:17.520898 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:17.521126 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:17.521232 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:17.521736 (+ 504us) server_negotiation.cc:299] Negotiation successful
0929 10:33:17.521900 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260929 10:33:17.523000 18320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:17.510133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:43605 (local address 127.15.82.65:43759)
0929 10:33:17.510634 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:17.510677 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:17.511945 (+ 1268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:17.512513 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:17.512521 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:17.512922 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:17.513578 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:17.513594 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:17.514464 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:17.514468 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:17.515444 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:17.515461 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:17.515972 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:17.515976 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:17.516143 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:17.516767 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:17.516783 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:17.518238 (+ 1455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:17.520071 (+ 1833us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:17.520077 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:17.520093 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:17.520356 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:17.520649 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:17.520652 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:17.520654 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:17.520796 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:17.521240 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:17.521249 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:17.521497 (+ 248us) client_negotiation.cc:769] Sending connection context
0929 10:33:17.521689 (+ 192us) client_negotiation.cc:240] Negotiation successful
0929 10:33:17.522022 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":97,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.66@KRBTEST.COM" created.
I20260929 10:33:17.524330 18318 heartbeater.cc:347] Connected to a master server at 127.15.82.126:43605
I20260929 10:33:17.524585 18318 heartbeater.cc:464] Registering TS with master...
I20260929 10:33:17.525079 18318 heartbeater.cc:511] Master 127.15.82.126:43605 requested a full tablet report, sending...
I20260929 10:33:17.526803 18127 ts_manager.cc:194] Registered new tserver with Master: e08abaccb3dd40e7b5360cf881e149e5 (127.15.82.65:42285)
I20260929 10:33:17.528512 18127 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.82.65@KRBTEST.COM'} at 127.15.82.65:43759
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab.
Entry for principal kudu/127.15.82.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab.
I20260929 10:33:17.541657 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.66:0
--local_ip_for_outbound_sockets=127.15.82.66
--webserver_interface=127.15.82.66
--webserver_port=0
--tserver_master_addrs=127.15.82.126:43605
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab
--principal=kudu/127.15.82.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:36819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:33:17.655678 18323 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:33:17.655946 18323 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:33:17.655988 18323 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:33:17.660203 18323 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:33:17.660265 18323 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:33:17.660331 18323 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.66
I20260929 10:33:17.665783 18323 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:36819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab
--principal=kudu/127.15.82.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.82.66
--webserver_port=0
--tserver_master_addrs=127.15.82.126:43605
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18323
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.66
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:33:17.667030 18323 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:33:17.668025 18323 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:33:17.677867 18329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:17.677867 18331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:17.677920 18328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:17.678303 18323 server_base.cc:1034] running on GCE node
I20260929 10:33:17.678754 18323 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:33:17.679461 18323 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:33:17.680629 18323 hybrid_clock.cc:648] HybridClock initialized: now 1790677997680605 us; error 34 us; skew 500 ppm
Sep 29 10:33:17 dist-test-slave-vg89 krb5kdc[18082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790677997, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:33:17.683568 18323 init.cc:377] Logged in from keytab as kudu/127.15.82.66@KRBTEST.COM (short username kudu)
I20260929 10:33:17.684444 18323 webserver.cc:460] Webserver started at http://127.15.82.66:36283/ using document root <none> and password file <none>
I20260929 10:33:17.685024 18323 fs_manager.cc:359] Metadata directory not provided
I20260929 10:33:17.685117 18323 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:33:17.685350 18323 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:33:17.687813 18323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-1/data/instance:
uuid: "6f08e63a43d84364a361b29601cf5fe3"
format_stamp: "Formatted at 2026-09-29 10:33:17 on dist-test-slave-vg89"
server_key: "b62df0284e5763217b098e6b12fa356c"
server_key_iv: "b314734b9f3dbb7022c6234c1110a572"
server_key_version: "encryptionkey@0"
I20260929 10:33:17.688496 18323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-1/wal/instance:
uuid: "6f08e63a43d84364a361b29601cf5fe3"
format_stamp: "Formatted at 2026-09-29 10:33:17 on dist-test-slave-vg89"
server_key: "b62df0284e5763217b098e6b12fa356c"
server_key_iv: "b314734b9f3dbb7022c6234c1110a572"
server_key_version: "encryptionkey@0"
I20260929 10:33:17.692862 18323 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 10:33:17.695736 18338 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:17.697003 18323 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260929 10:33:17.697146 18323 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-1/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-1/wal
uuid: "6f08e63a43d84364a361b29601cf5fe3"
format_stamp: "Formatted at 2026-09-29 10:33:17 on dist-test-slave-vg89"
server_key: "b62df0284e5763217b098e6b12fa356c"
server_key_iv: "b314734b9f3dbb7022c6234c1110a572"
server_key_version: "encryptionkey@0"
I20260929 10:33:17.697261 18323 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:33:17.715868 18323 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:33:17.719101 18323 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:33:17.719313 18323 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:33:17.719909 18323 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:33:17.720758 18323 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:33:17.720830 18323 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:17.720896 18323 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:33:17.720945 18323 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:17.729532 18323 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.66:36235
I20260929 10:33:17.729548 18451 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.66:36235 every 8 connection(s)
I20260929 10:33:17.730516 18323 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
I20260929 10:33:17.738921 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 18323
I20260929 10:33:17.739068 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-1/wal/instance
I20260929 10:33:17.739444 15689 external_mini_cluster.cc:1428] Setting key 9c07da02647d490b5123a44138d01f46
Sep 29 10:33:17 dist-test-slave-vg89 krb5kdc[18082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790677997, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.66@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:33:17.745067 18166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:17.732630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:58009 (local address 127.15.82.126:43605)
0929 10:33:17.732834 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:17.732838 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:17.733726 (+ 888us) server_negotiation.cc:407] Connection header received
0929 10:33:17.735090 (+ 1364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:17.735094 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:17.735155 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:17.735227 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:17.736404 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:17.736951 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:17.738095 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:17.738325 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:17.738463 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:17.741381 (+ 2918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:17.741385 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:17.741387 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:17.741410 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:17.742944 (+ 1534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:17.743447 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:17.743451 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:17.743453 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:17.743537 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:17.743887 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:17.743891 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:17.743892 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:17.744078 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:17.744190 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:17.744770 (+ 580us) server_negotiation.cc:299] Negotiation successful
0929 10:33:17.744907 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260929 10:33:17.745915 18454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:17.733070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:43605 (local address 127.15.82.66:58009)
0929 10:33:17.733548 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:17.733592 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:17.734865 (+ 1273us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:17.735411 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:17.735418 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:17.735772 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:17.736252 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:17.736265 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:17.737083 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:17.737088 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:17.737970 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:17.737979 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:17.738511 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:17.738514 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:17.738686 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:17.739308 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:17.739329 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:17.741216 (+ 1887us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:17.743056 (+ 1840us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:17.743062 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:17.743078 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:17.743335 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:17.743638 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:17.743641 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:17.743643 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:17.743786 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:17.744187 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:17.744194 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:17.744493 (+ 299us) client_negotiation.cc:769] Sending connection context
0929 10:33:17.744726 (+ 233us) client_negotiation.cc:240] Negotiation successful
0929 10:33:17.745060 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":126,"threads_started":1}
I20260929 10:33:17.747196 18452 heartbeater.cc:347] Connected to a master server at 127.15.82.126:43605
I20260929 10:33:17.747457 18452 heartbeater.cc:464] Registering TS with master...
I20260929 10:33:17.747949 18452 heartbeater.cc:511] Master 127.15.82.126:43605 requested a full tablet report, sending...
I20260929 10:33:17.749009 18127 ts_manager.cc:194] Registered new tserver with Master: 6f08e63a43d84364a361b29601cf5fe3 (127.15.82.66:36235)
WARNING: no policy specified for kudu/127.15.82.67@KRBTEST.COM; defaulting to no policy
I20260929 10:33:17.749640 18127 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.82.66@KRBTEST.COM'} at 127.15.82.66:58009
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab.
Entry for principal kudu/127.15.82.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab.
I20260929 10:33:17.767179 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.67:0
--local_ip_for_outbound_sockets=127.15.82.67
--webserver_interface=127.15.82.67
--webserver_port=0
--tserver_master_addrs=127.15.82.126:43605
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab
--principal=kudu/127.15.82.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:36819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:33:17.882158 18457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:33:17.882515 18457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:33:17.882568 18457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:33:17.886662 18457 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:33:17.886727 18457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:33:17.886790 18457 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.67
I20260929 10:33:17.892162 18457 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:36819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab
--principal=kudu/127.15.82.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.82.67
--webserver_port=0
--tserver_master_addrs=127.15.82.126:43605
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18457
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.67
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:33:17.893224 18457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:33:17.894234 18457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:33:17.904052 18465 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:17.904059 18462 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:17.904134 18457 server_base.cc:1034] running on GCE node
W20260929 10:33:17.904052 18463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:17.904742 18457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:33:17.905337 18457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:33:17.906473 18457 hybrid_clock.cc:648] HybridClock initialized: now 1790677997906451 us; error 28 us; skew 500 ppm
Sep 29 10:33:17 dist-test-slave-vg89 krb5kdc[18082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790677997, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:33:17.909540 18457 init.cc:377] Logged in from keytab as kudu/127.15.82.67@KRBTEST.COM (short username kudu)
I20260929 10:33:17.910475 18457 webserver.cc:460] Webserver started at http://127.15.82.67:46015/ using document root <none> and password file <none>
I20260929 10:33:17.911041 18457 fs_manager.cc:359] Metadata directory not provided
I20260929 10:33:17.911129 18457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:33:17.911367 18457 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:33:17.913551 18457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-2/data/instance:
uuid: "f95cba58857f43b6bf957de903065c1b"
format_stamp: "Formatted at 2026-09-29 10:33:17 on dist-test-slave-vg89"
server_key: "51a1c9b6b47c6e5be595db65b600cff7"
server_key_iv: "f1f91070b519ab7b75ea9d1b0fe955ca"
server_key_version: "encryptionkey@0"
I20260929 10:33:17.914188 18457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-2/wal/instance:
uuid: "f95cba58857f43b6bf957de903065c1b"
format_stamp: "Formatted at 2026-09-29 10:33:17 on dist-test-slave-vg89"
server_key: "51a1c9b6b47c6e5be595db65b600cff7"
server_key_iv: "f1f91070b519ab7b75ea9d1b0fe955ca"
server_key_version: "encryptionkey@0"
I20260929 10:33:17.917999 18457 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260929 10:33:17.920528 18472 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:17.921748 18457 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 10:33:17.921883 18457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-2/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-2/wal
uuid: "f95cba58857f43b6bf957de903065c1b"
format_stamp: "Formatted at 2026-09-29 10:33:17 on dist-test-slave-vg89"
server_key: "51a1c9b6b47c6e5be595db65b600cff7"
server_key_iv: "f1f91070b519ab7b75ea9d1b0fe955ca"
server_key_version: "encryptionkey@0"
I20260929 10:33:17.921998 18457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:33:17.937456 18457 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:33:17.941039 18457 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:33:17.941291 18457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:33:17.941920 18457 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:33:17.942924 18457 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:33:17.943044 18457 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:17.943114 18457 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:33:17.943156 18457 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:17.952127 18457 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.67:34125
I20260929 10:33:17.952149 18585 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.67:34125 every 8 connection(s)
I20260929 10:33:17.953078 18457 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
I20260929 10:33:17.953976 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 18457
I20260929 10:33:17.954099 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-2/wal/instance
I20260929 10:33:17.954444 15689 external_mini_cluster.cc:1428] Setting key 7b8be39c9e564471cfbff14f9c2ae5dd
Sep 29 10:33:17 dist-test-slave-vg89 krb5kdc[18082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790677997, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.67@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:33:17.966745 18166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:17.955337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.67:52157 (local address 127.15.82.126:43605)
0929 10:33:17.955455 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:17.955460 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:17.956541 (+ 1081us) server_negotiation.cc:407] Connection header received
0929 10:33:17.957564 (+ 1023us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:17.957567 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:17.957643 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:17.957725 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:17.958991 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:17.959515 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:17.960640 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:17.960935 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:17.961014 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:17.963450 (+ 2436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:17.963453 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:17.963455 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:17.963478 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:17.964951 (+ 1473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:17.965400 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:17.965402 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:17.965404 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:17.965476 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:17.965754 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:17.965756 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:17.965758 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:17.965935 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:17.966008 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:17.966523 (+ 515us) server_negotiation.cc:299] Negotiation successful
0929 10:33:17.966621 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260929 10:33:17.967873 18588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:17.955692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:43605 (local address 127.15.82.67:52157)
0929 10:33:17.956371 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:17.956423 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:17.957373 (+ 950us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:17.957914 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:17.957921 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:17.958289 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:17.958866 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:17.958882 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:17.959624 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:17.959626 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:17.960535 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:17.960544 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:17.961069 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:17.961072 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:17.961212 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:17.961767 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:17.961826 (+ 59us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:17.963295 (+ 1469us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:17.965046 (+ 1751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:17.965050 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:17.965062 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:17.965307 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:17.965551 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:17.965553 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:17.965554 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:17.965679 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:17.966020 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:17.966025 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:17.966253 (+ 228us) client_negotiation.cc:769] Sending connection context
0929 10:33:17.966493 (+ 240us) client_negotiation.cc:240] Negotiation successful
0929 10:33:17.966763 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":114,"threads_started":1}
I20260929 10:33:17.969013 18586 heartbeater.cc:347] Connected to a master server at 127.15.82.126:43605
I20260929 10:33:17.969228 18586 heartbeater.cc:464] Registering TS with master...
I20260929 10:33:17.969681 18586 heartbeater.cc:511] Master 127.15.82.126:43605 requested a full tablet report, sending...
I20260929 10:33:17.970674 18126 ts_manager.cc:194] Registered new tserver with Master: f95cba58857f43b6bf957de903065c1b (127.15.82.67:34125)
I20260929 10:33:17.971278 18126 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.82.67@KRBTEST.COM'} at 127.15.82.67:52157
I20260929 10:33:17.980734 15689 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260929 10:33:17.994514 18166 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:33:17.992924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57792 (local address 127.15.82.126:43605)
0929 10:33:17.993063 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:17.993067 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:17.993298 (+ 231us) server_negotiation.cc:407] Connection header received
0929 10:33:17.993412 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:17.993414 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:17.993463 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:17.993535 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:17.994278 (+ 743us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57792: BlockingRecv error: recv got EOF from 127.0.0.1:57792 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260929 10:33:17.995575 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 18189
I20260929 10:33:18.001185 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 18323
I20260929 10:33:18.006376 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 18457
I20260929 10:33:18.011456 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 18095
2026-09-29T10:33:18Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2182 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790677981592896-15689-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:33:18 dist-test-slave-vg89 krb5kdc[18598](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:33:18 dist-test-slave-vg89 krb5kdc[18598](info): set up 2 sockets
Sep 29 10:33:18 dist-test-slave-vg89 krb5kdc[18598](info): commencing operation
krb5kdc: starting...
W20260929 10:33:19.102655 15689 mini_kdc.cc:121] Time spent starting KDC: real 1.069s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Sep 29 10:33:19 dist-test-slave-vg89 krb5kdc[18598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790677999, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:33:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:33:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.82.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab.
Entry for principal kudu/127.15.82.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab.
I20260929 10:33:19.223630 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790677981592896-15689-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790677981592896-15689-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790677981592896-15689-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:39437
--webserver_interface=127.15.82.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab
--principal=kudu/127.15.82.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:38831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.82.126:39437
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:33:19.336083 18611 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:33:19.336395 18611 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:33:19.336473 18611 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:33:19.340495 18611 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:33:19.340581 18611 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:33:19.340620 18611 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:33:19.340675 18611 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:33:19.340716 18611 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:33:19.346010 18611 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:38831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790677981592896-15689-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790677981592896-15689-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.82.126:39437
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab
--principal=kudu/127.15.82.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:39437
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.82.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18611
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790677981592896-15689-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:33:19.347113 18611 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:33:19.348176 18611 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:33:19.357138 18617 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:19.357133 18619 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:19.357133 18616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:19.357404 18611 server_base.cc:1034] running on GCE node
I20260929 10:33:19.358040 18611 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:33:19.359285 18611 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:33:19.360466 18611 hybrid_clock.cc:648] HybridClock initialized: now 1790677999360447 us; error 30 us; skew 500 ppm
Sep 29 10:33:19 dist-test-slave-vg89 krb5kdc[18598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790677999, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:33:19.366837 18611 init.cc:377] Logged in from keytab as kudu/127.15.82.126@KRBTEST.COM (short username kudu)
I20260929 10:33:19.367751 18611 webserver.cc:460] Webserver started at http://127.15.82.126:41507/ using document root <none> and password file <none>
I20260929 10:33:19.368331 18611 fs_manager.cc:359] Metadata directory not provided
I20260929 10:33:19.368423 18611 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:33:19.368655 18611 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:33:19.370828 18611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790677981592896-15689-0/minicluster-data/master-0/data/instance:
uuid: "0d68ea02898d464e9c21c63b9a9981ef"
format_stamp: "Formatted at 2026-09-29 10:33:19 on dist-test-slave-vg89"
server_key: "177aa7f115605af8f75db3509214e6c8"
server_key_iv: "0403630422caeeb22978df17d98a6dc1"
server_key_version: "encryptionkey@0"
I20260929 10:33:19.371403 18611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790677981592896-15689-0/minicluster-data/master-0/wal/instance:
uuid: "0d68ea02898d464e9c21c63b9a9981ef"
format_stamp: "Formatted at 2026-09-29 10:33:19 on dist-test-slave-vg89"
server_key: "177aa7f115605af8f75db3509214e6c8"
server_key_iv: "0403630422caeeb22978df17d98a6dc1"
server_key_version: "encryptionkey@0"
I20260929 10:33:19.375147 18611 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260929 10:33:19.377542 18626 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:19.378758 18611 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 10:33:19.378898 18611 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790677981592896-15689-0/minicluster-data/master-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790677981592896-15689-0/minicluster-data/master-0/wal
uuid: "0d68ea02898d464e9c21c63b9a9981ef"
format_stamp: "Formatted at 2026-09-29 10:33:19 on dist-test-slave-vg89"
server_key: "177aa7f115605af8f75db3509214e6c8"
server_key_iv: "0403630422caeeb22978df17d98a6dc1"
server_key_version: "encryptionkey@0"
I20260929 10:33:19.379007 18611 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790677981592896-15689-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790677981592896-15689-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790677981592896-15689-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:33:19.389823 18611 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:33:19.392630 18611 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:33:19.392838 18611 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:33:19.400619 18611 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.126:39437
I20260929 10:33:19.400897 18678 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.126:39437 every 8 connection(s)
I20260929 10:33:19.401654 18611 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
I20260929 10:33:19.404430 18679 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:33:19.409919 18679 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d68ea02898d464e9c21c63b9a9981ef: Bootstrap starting.
I20260929 10:33:19.410156 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 18611
I20260929 10:33:19.410287 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790677981592896-15689-0/minicluster-data/master-0/wal/instance
I20260929 10:33:19.410604 15689 external_mini_cluster.cc:1428] Setting key 3d508ddb3f4a70d2dd77997ab83ecce2
I20260929 10:33:19.412634 18679 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0d68ea02898d464e9c21c63b9a9981ef: Neither blocks nor log segments found. Creating new log.
I20260929 10:33:19.413384 18679 log.cc:824] T 00000000000000000000000000000000 P 0d68ea02898d464e9c21c63b9a9981ef: Log is configured to *not* fsync() on all Append() calls
I20260929 10:33:19.415278 18679 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d68ea02898d464e9c21c63b9a9981ef: No bootstrap required, opened a new log
I20260929 10:33:19.418700 18679 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0d68ea02898d464e9c21c63b9a9981ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d68ea02898d464e9c21c63b9a9981ef" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 39437 } }
I20260929 10:33:19.418991 18679 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0d68ea02898d464e9c21c63b9a9981ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:33:19.419040 18679 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0d68ea02898d464e9c21c63b9a9981ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d68ea02898d464e9c21c63b9a9981ef, State: Initialized, Role: FOLLOWER
Sep 29 10:33:19 dist-test-slave-vg89 krb5kdc[18598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790677999, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:33:19.419529 18679 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0d68ea02898d464e9c21c63b9a9981ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d68ea02898d464e9c21c63b9a9981ef" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 39437 } }
I20260929 10:33:19.419692 18679 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0d68ea02898d464e9c21c63b9a9981ef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:33:19.419765 18679 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0d68ea02898d464e9c21c63b9a9981ef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:33:19.419878 18679 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0d68ea02898d464e9c21c63b9a9981ef [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:33:19.420766 18679 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0d68ea02898d464e9c21c63b9a9981ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d68ea02898d464e9c21c63b9a9981ef" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 39437 } }
I20260929 10:33:19.421100 18679 leader_election.cc:302] T 00000000000000000000000000000000 P 0d68ea02898d464e9c21c63b9a9981ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0d68ea02898d464e9c21c63b9a9981ef; no voters:
I20260929 10:33:19.421458 18679 leader_election.cc:288] T 00000000000000000000000000000000 P 0d68ea02898d464e9c21c63b9a9981ef [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:33:19.421579 18684 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0d68ea02898d464e9c21c63b9a9981ef [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:33:19.421784 18684 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0d68ea02898d464e9c21c63b9a9981ef [term 1 LEADER]: Becoming Leader. State: Replica: 0d68ea02898d464e9c21c63b9a9981ef, State: Running, Role: LEADER
I20260929 10:33:19.422142 18684 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0d68ea02898d464e9c21c63b9a9981ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d68ea02898d464e9c21c63b9a9981ef" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 39437 } }
I20260929 10:33:19.422828 18679 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0d68ea02898d464e9c21c63b9a9981ef [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:33:19.424038 18685 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d68ea02898d464e9c21c63b9a9981ef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0d68ea02898d464e9c21c63b9a9981ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d68ea02898d464e9c21c63b9a9981ef" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 39437 } } }
I20260929 10:33:19.424168 18685 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d68ea02898d464e9c21c63b9a9981ef [sys.catalog]: This master's current role is: LEADER
I20260929 10:33:19.424697 18693 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:33:19.424036 18686 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d68ea02898d464e9c21c63b9a9981ef [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d68ea02898d464e9c21c63b9a9981ef. Latest consensus state: current_term: 1 leader_uuid: "0d68ea02898d464e9c21c63b9a9981ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d68ea02898d464e9c21c63b9a9981ef" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 39437 } } }
I20260929 10:33:19.424860 18686 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d68ea02898d464e9c21c63b9a9981ef [sys.catalog]: This master's current role is: LEADER
I20260929 10:33:19.426328 18682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:19.411891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55602 (local address 127.15.82.126:39437)
0929 10:33:19.412322 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:19.412337 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:19.412380 (+ 43us) server_negotiation.cc:407] Connection header received
0929 10:33:19.413059 (+ 679us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:19.413078 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:19.413431 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:19.413804 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:19.414732 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:19.415767 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:19.416653 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:19.417128 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:19.417295 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:19.420093 (+ 2798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:19.420103 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:19.420116 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:19.420144 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:19.422272 (+ 2128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:19.422784 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:19.422790 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:19.422795 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:19.422946 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:19.423267 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:19.423270 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:19.423272 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:19.423814 (+ 542us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:19.424025 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:19.425085 (+ 1060us) server_negotiation.cc:299] Negotiation successful
0929 10:33:19.425394 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":136,"threads_started":1}
I20260929 10:33:19.427944 18693 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:33:19.429553 15689 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 10:33:19.434442 18693 catalog_manager.cc:1347] Generated new cluster ID: f7137cb9a6fc4b638d27e036d58d39b0
I20260929 10:33:19.434501 18693 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:33:19.447767 18682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:19.442739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55608 (local address 127.15.82.126:39437)
0929 10:33:19.442849 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:19.442854 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:19.443223 (+ 369us) server_negotiation.cc:407] Connection header received
0929 10:33:19.443393 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:19.443396 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:19.443451 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:19.443533 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:19.444090 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:19.444617 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:19.445465 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:19.445718 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:19.445824 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:19.446740 (+ 916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:19.446744 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:19.446746 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:33:19.447267 (+ 521us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:19.447300 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:33:19.447317 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:19.447425 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:19.447546 (+ 121us) server_negotiation.cc:299] Negotiation successful
0929 10:33:19.447614 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260929 10:33:19.452086 18693 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:33:19.453716 18693 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:33:19.464083 18693 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0d68ea02898d464e9c21c63b9a9981ef: Generated new TSK 0
I20260929 10:33:19.464884 18693 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 10:33:19.472486 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 18611
2026-09-29T10:33:19Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1459 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:33:19 dist-test-slave-vg89 krb5kdc[18712](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:33:19 dist-test-slave-vg89 krb5kdc[18712](info): set up 2 sockets
Sep 29 10:33:19 dist-test-slave-vg89 krb5kdc[18712](info): commencing operation
krb5kdc: starting...
W20260929 10:33:20.510809 15689 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Sep 29 10:33:20 dist-test-slave-vg89 krb5kdc[18712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678000, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:33:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:33:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.82.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab.
Entry for principal kudu/127.15.82.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab.
I20260929 10:33:20.641738 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:41399
--webserver_interface=127.15.82.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab
--principal=kudu/127.15.82.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:34201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.82.126:41399
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:33:20.754091 18725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:33:20.754361 18725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:33:20.754475 18725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:33:20.758589 18725 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:33:20.758649 18725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:33:20.758668 18725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:33:20.758685 18725 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:33:20.758700 18725 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:33:20.764048 18725 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:34201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.82.126:41399
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab
--principal=kudu/127.15.82.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:41399
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.82.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18725
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:33:20.765130 18725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:33:20.766202 18725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:33:20.774919 18733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:20.774987 18731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:20.774964 18730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:20.775069 18725 server_base.cc:1034] running on GCE node
I20260929 10:33:20.775878 18725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:33:20.776960 18725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:33:20.778116 18725 hybrid_clock.cc:648] HybridClock initialized: now 1790678000778079 us; error 48 us; skew 500 ppm
Sep 29 10:33:20 dist-test-slave-vg89 krb5kdc[18712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678000, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:33:20.780925 18725 init.cc:377] Logged in from keytab as kudu/127.15.82.126@KRBTEST.COM (short username kudu)
I20260929 10:33:20.781843 18725 webserver.cc:460] Webserver started at http://127.15.82.126:35367/ using document root <none> and password file <none>
I20260929 10:33:20.782464 18725 fs_manager.cc:359] Metadata directory not provided
I20260929 10:33:20.782555 18725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:33:20.782785 18725 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:33:20.784778 18725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/minicluster-data/master-0/data/instance:
uuid: "a7df252f197043968d7f1e41e5cf88ac"
format_stamp: "Formatted at 2026-09-29 10:33:20 on dist-test-slave-vg89"
server_key: "c7bcb96b7f6e642046ae78d13a3d5f84"
server_key_iv: "fea52b1d81a18784204789be74e2b3c7"
server_key_version: "encryptionkey@0"
I20260929 10:33:20.785342 18725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/minicluster-data/master-0/wal/instance:
uuid: "a7df252f197043968d7f1e41e5cf88ac"
format_stamp: "Formatted at 2026-09-29 10:33:20 on dist-test-slave-vg89"
server_key: "c7bcb96b7f6e642046ae78d13a3d5f84"
server_key_iv: "fea52b1d81a18784204789be74e2b3c7"
server_key_version: "encryptionkey@0"
I20260929 10:33:20.789083 18725 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260929 10:33:20.791540 18740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:20.792786 18725 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 10:33:20.792943 18725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/minicluster-data/master-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/minicluster-data/master-0/wal
uuid: "a7df252f197043968d7f1e41e5cf88ac"
format_stamp: "Formatted at 2026-09-29 10:33:20 on dist-test-slave-vg89"
server_key: "c7bcb96b7f6e642046ae78d13a3d5f84"
server_key_iv: "fea52b1d81a18784204789be74e2b3c7"
server_key_version: "encryptionkey@0"
I20260929 10:33:20.793054 18725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:33:20.803179 18725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:33:20.806164 18725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:33:20.806370 18725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:33:20.813331 18725 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.126:41399
I20260929 10:33:20.813611 18792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.126:41399 every 8 connection(s)
I20260929 10:33:20.814302 18725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
I20260929 10:33:20.817111 18793 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:33:20.818634 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 18725
I20260929 10:33:20.818771 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/minicluster-data/master-0/wal/instance
I20260929 10:33:20.819060 15689 external_mini_cluster.cc:1428] Setting key ed96934155444e0a6c8452fb101775ae
I20260929 10:33:20.823443 18793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7df252f197043968d7f1e41e5cf88ac: Bootstrap starting.
I20260929 10:33:20.826041 18793 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a7df252f197043968d7f1e41e5cf88ac: Neither blocks nor log segments found. Creating new log.
I20260929 10:33:20.827019 18793 log.cc:824] T 00000000000000000000000000000000 P a7df252f197043968d7f1e41e5cf88ac: Log is configured to *not* fsync() on all Append() calls
Sep 29 10:33:20 dist-test-slave-vg89 krb5kdc[18712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678000, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:33:20.829267 18793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7df252f197043968d7f1e41e5cf88ac: No bootstrap required, opened a new log
I20260929 10:33:20.834095 18796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:20.820491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48690 (local address 127.15.82.126:41399)
0929 10:33:20.820894 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:20.820910 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:20.820945 (+ 35us) server_negotiation.cc:407] Connection header received
0929 10:33:20.821574 (+ 629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:20.821591 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:20.821921 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:20.822189 (+ 268us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:20.823134 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:20.824159 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:20.825373 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:20.825828 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:20.825950 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:20.828922 (+ 2972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:20.828930 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:20.828949 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:20.828979 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:20.830969 (+ 1990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:20.831392 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:20.831398 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:20.831404 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:20.831555 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:20.831830 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:20.831835 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:20.831838 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:20.832383 (+ 545us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:20.832569 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:20.832819 (+ 250us) server_negotiation.cc:299] Negotiation successful
0929 10:33:20.833116 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":130,"threads_started":1}
I20260929 10:33:20.834071 18793 raft_consensus.cc:348] T 00000000000000000000000000000000 P a7df252f197043968d7f1e41e5cf88ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7df252f197043968d7f1e41e5cf88ac" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 41399 } }
I20260929 10:33:20.834367 18793 raft_consensus.cc:374] T 00000000000000000000000000000000 P a7df252f197043968d7f1e41e5cf88ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:33:20.834448 18793 raft_consensus.cc:729] T 00000000000000000000000000000000 P a7df252f197043968d7f1e41e5cf88ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7df252f197043968d7f1e41e5cf88ac, State: Initialized, Role: FOLLOWER
I20260929 10:33:20.834851 18793 consensus_queue.cc:260] T 00000000000000000000000000000000 P a7df252f197043968d7f1e41e5cf88ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7df252f197043968d7f1e41e5cf88ac" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 41399 } }
I20260929 10:33:20.834957 18793 raft_consensus.cc:388] T 00000000000000000000000000000000 P a7df252f197043968d7f1e41e5cf88ac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:33:20.834995 18793 raft_consensus.cc:482] T 00000000000000000000000000000000 P a7df252f197043968d7f1e41e5cf88ac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:33:20.835057 18793 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a7df252f197043968d7f1e41e5cf88ac [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:33:20.836148 18793 raft_consensus.cc:504] T 00000000000000000000000000000000 P a7df252f197043968d7f1e41e5cf88ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7df252f197043968d7f1e41e5cf88ac" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 41399 } }
I20260929 10:33:20.836485 18793 leader_election.cc:302] T 00000000000000000000000000000000 P a7df252f197043968d7f1e41e5cf88ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a7df252f197043968d7f1e41e5cf88ac; no voters:
I20260929 10:33:20.836794 18793 leader_election.cc:288] T 00000000000000000000000000000000 P a7df252f197043968d7f1e41e5cf88ac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:33:20.836918 18798 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a7df252f197043968d7f1e41e5cf88ac [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:33:20.837143 18798 raft_consensus.cc:686] T 00000000000000000000000000000000 P a7df252f197043968d7f1e41e5cf88ac [term 1 LEADER]: Becoming Leader. State: Replica: a7df252f197043968d7f1e41e5cf88ac, State: Running, Role: LEADER
I20260929 10:33:20.837456 18798 consensus_queue.cc:237] T 00000000000000000000000000000000 P a7df252f197043968d7f1e41e5cf88ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7df252f197043968d7f1e41e5cf88ac" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 41399 } }
I20260929 10:33:20.838018 18793 sys_catalog.cc:564] T 00000000000000000000000000000000 P a7df252f197043968d7f1e41e5cf88ac [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:33:20.839051 18800 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7df252f197043968d7f1e41e5cf88ac [sys.catalog]: SysCatalogTable state changed. Reason: New leader a7df252f197043968d7f1e41e5cf88ac. Latest consensus state: current_term: 1 leader_uuid: "a7df252f197043968d7f1e41e5cf88ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7df252f197043968d7f1e41e5cf88ac" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 41399 } } }
I20260929 10:33:20.839191 18800 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7df252f197043968d7f1e41e5cf88ac [sys.catalog]: This master's current role is: LEADER
I20260929 10:33:20.839515 18799 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7df252f197043968d7f1e41e5cf88ac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a7df252f197043968d7f1e41e5cf88ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7df252f197043968d7f1e41e5cf88ac" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 41399 } } }
I20260929 10:33:20.839646 18799 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7df252f197043968d7f1e41e5cf88ac [sys.catalog]: This master's current role is: LEADER
I20260929 10:33:20.840375 18807 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:33:20.843046 18807 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:33:20.848634 18807 catalog_manager.cc:1347] Generated new cluster ID: 47be3fd287214ebab83b10e5e202c6fd
I20260929 10:33:20.848701 18807 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:33:20.863147 18807 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:33:20.864441 18807 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:33:20.874136 18807 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a7df252f197043968d7f1e41e5cf88ac: Generated new TSK 0
I20260929 10:33:20.875048 18807 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.82.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab.
Entry for principal kudu/127.15.82.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab.
I20260929 10:33:20.905898 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.65:0
--local_ip_for_outbound_sockets=127.15.82.65
--webserver_interface=127.15.82.65
--webserver_port=0
--tserver_master_addrs=127.15.82.126:41399
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab
--principal=kudu/127.15.82.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:34201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:33:21.023452 18819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:33:21.023790 18819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:33:21.023870 18819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:33:21.028056 18819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:33:21.028177 18819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:33:21.028277 18819 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.65
I20260929 10:33:21.033984 18819 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:34201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab
--principal=kudu/127.15.82.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.82.65
--webserver_port=0
--tserver_master_addrs=127.15.82.126:41399
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18819
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.65
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:33:21.035202 18819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:33:21.036298 18819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:33:21.047494 18824 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:21.047489 18827 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:21.047511 18825 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:21.047906 18819 server_base.cc:1034] running on GCE node
I20260929 10:33:21.048352 18819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:33:21.049059 18819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:33:21.050259 18819 hybrid_clock.cc:648] HybridClock initialized: now 1790678001050248 us; error 31 us; skew 500 ppm
Sep 29 10:33:21 dist-test-slave-vg89 krb5kdc[18712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678001, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:33:21.054301 18819 init.cc:377] Logged in from keytab as kudu/127.15.82.65@KRBTEST.COM (short username kudu)
I20260929 10:33:21.055932 18819 webserver.cc:460] Webserver started at http://127.15.82.65:39171/ using document root <none> and password file <none>
I20260929 10:33:21.062885 18819 fs_manager.cc:359] Metadata directory not provided
I20260929 10:33:21.063258 18819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:33:21.063596 18819 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:33:21.067479 18819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-0/data/instance:
uuid: "11367f0bd8ed4eec9650ea64e8ae4efa"
format_stamp: "Formatted at 2026-09-29 10:33:21 on dist-test-slave-vg89"
server_key: "c3e30921429a26f16683bfe0c24ee6e4"
server_key_iv: "c1b5282c617482ce300a4181e97c94db"
server_key_version: "encryptionkey@0"
I20260929 10:33:21.068288 18819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-0/wal/instance:
uuid: "11367f0bd8ed4eec9650ea64e8ae4efa"
format_stamp: "Formatted at 2026-09-29 10:33:21 on dist-test-slave-vg89"
server_key: "c3e30921429a26f16683bfe0c24ee6e4"
server_key_iv: "c1b5282c617482ce300a4181e97c94db"
server_key_version: "encryptionkey@0"
I20260929 10:33:21.074735 18819 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260929 10:33:21.079862 18834 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:21.081848 18819 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260929 10:33:21.082047 18819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-0/wal
uuid: "11367f0bd8ed4eec9650ea64e8ae4efa"
format_stamp: "Formatted at 2026-09-29 10:33:21 on dist-test-slave-vg89"
server_key: "c3e30921429a26f16683bfe0c24ee6e4"
server_key_iv: "c1b5282c617482ce300a4181e97c94db"
server_key_version: "encryptionkey@0"
I20260929 10:33:21.082223 18819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:33:21.100598 18819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:33:21.104014 18819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:33:21.104192 18819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:33:21.104776 18819 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:33:21.105626 18819 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:33:21.105674 18819 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:21.105715 18819 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:33:21.105731 18819 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:21.115347 18819 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.65:37767
I20260929 10:33:21.115358 18947 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.65:37767 every 8 connection(s)
I20260929 10:33:21.116297 18819 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
I20260929 10:33:21.123909 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 18819
I20260929 10:33:21.124061 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-0/wal/instance
I20260929 10:33:21.124392 15689 external_mini_cluster.cc:1428] Setting key e9c9230b68b00cdb4ca995cae864ccce
Sep 29 10:33:21 dist-test-slave-vg89 krb5kdc[18712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678001, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.65@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:33:21.134104 18796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:21.118176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.65:39429 (local address 127.15.82.126:41399)
0929 10:33:21.118331 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:21.118336 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:21.119393 (+ 1057us) server_negotiation.cc:407] Connection header received
0929 10:33:21.120719 (+ 1326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:21.120722 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:21.120771 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:21.120853 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:21.122077 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:21.122686 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:21.123905 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:21.124274 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:21.124407 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:21.127291 (+ 2884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:21.127297 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:21.127303 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:21.127346 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:21.129587 (+ 2241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:21.131651 (+ 2064us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:21.131658 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:21.131662 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:21.131784 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:21.132321 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:21.132327 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:21.132331 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:21.132651 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:21.132745 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:21.133653 (+ 908us) server_negotiation.cc:299] Negotiation successful
0929 10:33:21.133853 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260929 10:33:21.136674 18950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:21.118669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:41399 (local address 127.15.82.65:39429)
0929 10:33:21.119182 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:21.119262 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:21.120513 (+ 1251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:21.121020 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:21.121026 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:21.121443 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:21.121939 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:21.121953 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:21.122826 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:21.122829 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:21.123769 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:21.123778 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:21.124481 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:21.124489 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:21.124664 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:21.125323 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:21.125340 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:21.127094 (+ 1754us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:21.129721 (+ 2627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:21.129730 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:21.129752 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:21.130158 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:21.131923 (+ 1765us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:21.131927 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:21.131931 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:21.132178 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:21.132852 (+ 674us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:21.132861 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:21.133285 (+ 424us) client_negotiation.cc:769] Sending connection context
0929 10:33:21.134818 (+ 1533us) client_negotiation.cc:240] Negotiation successful
0929 10:33:21.135299 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":130,"threads_started":1}
WARNING: no policy specified for kudu/127.15.82.66@KRBTEST.COM; defaulting to no policy
I20260929 10:33:21.138655 18948 heartbeater.cc:347] Connected to a master server at 127.15.82.126:41399
I20260929 10:33:21.138969 18948 heartbeater.cc:464] Registering TS with master...
I20260929 10:33:21.139719 18948 heartbeater.cc:511] Master 127.15.82.126:41399 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.66@KRBTEST.COM" created.
I20260929 10:33:21.141884 18757 ts_manager.cc:194] Registered new tserver with Master: 11367f0bd8ed4eec9650ea64e8ae4efa (127.15.82.65:37767)
I20260929 10:33:21.143726 18757 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.82.65@KRBTEST.COM'} at 127.15.82.65:39429
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab.
Entry for principal kudu/127.15.82.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab.
I20260929 10:33:21.157598 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.66:0
--local_ip_for_outbound_sockets=127.15.82.66
--webserver_interface=127.15.82.66
--webserver_port=0
--tserver_master_addrs=127.15.82.126:41399
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab
--principal=kudu/127.15.82.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:34201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:33:21.271306 18953 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:33:21.271589 18953 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:33:21.271636 18953 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:33:21.275913 18953 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:33:21.275974 18953 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:33:21.276067 18953 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.66
I20260929 10:33:21.281512 18953 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:34201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab
--principal=kudu/127.15.82.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.82.66
--webserver_port=0
--tserver_master_addrs=127.15.82.126:41399
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18953
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.66
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:33:21.282742 18953 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:33:21.283797 18953 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:33:21.293953 18959 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:21.293928 18961 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:21.293917 18958 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:21.293985 18953 server_base.cc:1034] running on GCE node
I20260929 10:33:21.294610 18953 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:33:21.295296 18953 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:33:21.296459 18953 hybrid_clock.cc:648] HybridClock initialized: now 1790678001296442 us; error 40 us; skew 500 ppm
Sep 29 10:33:21 dist-test-slave-vg89 krb5kdc[18712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678001, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:33:21.299461 18953 init.cc:377] Logged in from keytab as kudu/127.15.82.66@KRBTEST.COM (short username kudu)
I20260929 10:33:21.300406 18953 webserver.cc:460] Webserver started at http://127.15.82.66:37455/ using document root <none> and password file <none>
I20260929 10:33:21.300977 18953 fs_manager.cc:359] Metadata directory not provided
I20260929 10:33:21.301036 18953 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:33:21.301249 18953 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:33:21.303457 18953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-1/data/instance:
uuid: "acefdca187c44c07b3434e1d89af3d26"
format_stamp: "Formatted at 2026-09-29 10:33:21 on dist-test-slave-vg89"
server_key: "7686831d226ac184245e1f655d626ed1"
server_key_iv: "af8a14bb90dec6dcfc10a7af41a76ad0"
server_key_version: "encryptionkey@0"
I20260929 10:33:21.304025 18953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-1/wal/instance:
uuid: "acefdca187c44c07b3434e1d89af3d26"
format_stamp: "Formatted at 2026-09-29 10:33:21 on dist-test-slave-vg89"
server_key: "7686831d226ac184245e1f655d626ed1"
server_key_iv: "af8a14bb90dec6dcfc10a7af41a76ad0"
server_key_version: "encryptionkey@0"
I20260929 10:33:21.307998 18953 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 10:33:21.310715 18968 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:21.312450 18953 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 10:33:21.312574 18953 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-1/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-1/wal
uuid: "acefdca187c44c07b3434e1d89af3d26"
format_stamp: "Formatted at 2026-09-29 10:33:21 on dist-test-slave-vg89"
server_key: "7686831d226ac184245e1f655d626ed1"
server_key_iv: "af8a14bb90dec6dcfc10a7af41a76ad0"
server_key_version: "encryptionkey@0"
I20260929 10:33:21.312700 18953 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:33:21.338214 18953 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:33:21.341815 18953 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:33:21.342039 18953 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:33:21.342667 18953 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:33:21.343494 18953 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:33:21.343540 18953 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:21.343608 18953 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:33:21.343652 18953 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:21.352250 18953 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.66:37163
I20260929 10:33:21.352336 19081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.66:37163 every 8 connection(s)
I20260929 10:33:21.353209 18953 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
I20260929 10:33:21.354763 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 18953
I20260929 10:33:21.354887 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-1/wal/instance
I20260929 10:33:21.355217 15689 external_mini_cluster.cc:1428] Setting key 5caca9370840ebae0e74354f774844fb
Sep 29 10:33:21 dist-test-slave-vg89 krb5kdc[18712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678001, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.66@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.82.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.67@KRBTEST.COM" created.
I20260929 10:33:21.369271 18796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:21.355252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:54815 (local address 127.15.82.126:41399)
0929 10:33:21.355397 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:21.355401 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:21.356448 (+ 1047us) server_negotiation.cc:407] Connection header received
0929 10:33:21.357899 (+ 1451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:21.357902 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:21.357956 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:21.358036 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:21.359333 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:21.360225 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:21.361536 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:21.361986 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:21.362126 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:21.365090 (+ 2964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:21.365094 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:21.365096 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:21.365122 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:21.366707 (+ 1585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:21.367330 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:21.367333 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:21.367335 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:21.367411 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:21.367898 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:21.367901 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:21.367902 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:21.368142 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:21.368247 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:21.368867 (+ 620us) server_negotiation.cc:299] Negotiation successful
0929 10:33:21.369018 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260929 10:33:21.370086 19084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:21.355781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:41399 (local address 127.15.82.66:54815)
0929 10:33:21.356300 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:21.356354 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:21.357674 (+ 1320us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:21.358221 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:21.358228 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:21.358691 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:21.359187 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:21.359201 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:21.360388 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:21.360391 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:21.361401 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:21.361410 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:21.362154 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:21.362157 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:21.362316 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:21.363007 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:21.363025 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:21.364895 (+ 1870us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:21.366818 (+ 1923us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:21.366823 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:21.366838 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:21.367229 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:21.367516 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:21.367522 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:21.367527 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:21.367766 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:21.368302 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:21.368308 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:21.368576 (+ 268us) client_negotiation.cc:769] Sending connection context
0929 10:33:21.368774 (+ 198us) client_negotiation.cc:240] Negotiation successful
0929 10:33:21.369079 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":143,"threads_started":1}
I20260929 10:33:21.371465 19082 heartbeater.cc:347] Connected to a master server at 127.15.82.126:41399
I20260929 10:33:21.371722 19082 heartbeater.cc:464] Registering TS with master...
I20260929 10:33:21.372243 19082 heartbeater.cc:511] Master 127.15.82.126:41399 requested a full tablet report, sending...
I20260929 10:33:21.373385 18757 ts_manager.cc:194] Registered new tserver with Master: acefdca187c44c07b3434e1d89af3d26 (127.15.82.66:37163)
I20260929 10:33:21.374125 18757 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.82.66@KRBTEST.COM'} at 127.15.82.66:54815
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab.
Entry for principal kudu/127.15.82.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab.
I20260929 10:33:21.384920 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.67:0
--local_ip_for_outbound_sockets=127.15.82.67
--webserver_interface=127.15.82.67
--webserver_port=0
--tserver_master_addrs=127.15.82.126:41399
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab
--principal=kudu/127.15.82.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:34201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:33:21.498149 19087 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:33:21.498526 19087 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:33:21.498616 19087 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:33:21.503038 19087 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:33:21.503115 19087 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:33:21.503216 19087 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.67
I20260929 10:33:21.508863 19087 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:34201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab
--principal=kudu/127.15.82.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.82.67
--webserver_port=0
--tserver_master_addrs=127.15.82.126:41399
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19087
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.67
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:33:21.510047 19087 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:33:21.511220 19087 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:33:21.521369 19093 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:21.521358 19092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:21.521363 19095 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:21.521776 19087 server_base.cc:1034] running on GCE node
I20260929 10:33:21.522298 19087 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:33:21.522997 19087 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:33:21.524199 19087 hybrid_clock.cc:648] HybridClock initialized: now 1790678001524175 us; error 29 us; skew 500 ppm
Sep 29 10:33:21 dist-test-slave-vg89 krb5kdc[18712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678001, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:33:21.527047 19087 init.cc:377] Logged in from keytab as kudu/127.15.82.67@KRBTEST.COM (short username kudu)
I20260929 10:33:21.528005 19087 webserver.cc:460] Webserver started at http://127.15.82.67:44465/ using document root <none> and password file <none>
I20260929 10:33:21.528623 19087 fs_manager.cc:359] Metadata directory not provided
I20260929 10:33:21.528715 19087 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:33:21.528964 19087 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:33:21.531204 19087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-2/data/instance:
uuid: "a11d7a835bb24f5b9129f13f1e5dba30"
format_stamp: "Formatted at 2026-09-29 10:33:21 on dist-test-slave-vg89"
server_key: "e8e7a277a5234b5b2af56c3029a4e2f6"
server_key_iv: "b30b167c291d376d7b0d204756596624"
server_key_version: "encryptionkey@0"
I20260929 10:33:21.531827 19087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-2/wal/instance:
uuid: "a11d7a835bb24f5b9129f13f1e5dba30"
format_stamp: "Formatted at 2026-09-29 10:33:21 on dist-test-slave-vg89"
server_key: "e8e7a277a5234b5b2af56c3029a4e2f6"
server_key_iv: "b30b167c291d376d7b0d204756596624"
server_key_version: "encryptionkey@0"
I20260929 10:33:21.535866 19087 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 10:33:21.538599 19102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:21.540027 19087 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 10:33:21.540164 19087 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-2/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-2/wal
uuid: "a11d7a835bb24f5b9129f13f1e5dba30"
format_stamp: "Formatted at 2026-09-29 10:33:21 on dist-test-slave-vg89"
server_key: "e8e7a277a5234b5b2af56c3029a4e2f6"
server_key_iv: "b30b167c291d376d7b0d204756596624"
server_key_version: "encryptionkey@0"
I20260929 10:33:21.540292 19087 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:33:21.561916 19087 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:33:21.567432 19087 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:33:21.567679 19087 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:33:21.568315 19087 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:33:21.569198 19087 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:33:21.569250 19087 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:21.569293 19087 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:33:21.569308 19087 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:21.578970 19087 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.67:44721
I20260929 10:33:21.579001 19215 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.67:44721 every 8 connection(s)
I20260929 10:33:21.579985 19087 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
I20260929 10:33:21.582260 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 19087
I20260929 10:33:21.582427 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790677981592896-15689-0/minicluster-data/ts-2/wal/instance
I20260929 10:33:21.582778 15689 external_mini_cluster.cc:1428] Setting key c2cd885d8f09617100df461a038ec8dc
Sep 29 10:33:21 dist-test-slave-vg89 krb5kdc[18712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678001, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.67@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:33:21.595016 18796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:21.582025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.67:50271 (local address 127.15.82.126:41399)
0929 10:33:21.582221 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:21.582226 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:21.583339 (+ 1113us) server_negotiation.cc:407] Connection header received
0929 10:33:21.585121 (+ 1782us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:21.585125 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:21.585187 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:21.585269 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:21.586606 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:21.587164 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:21.588332 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:21.588571 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:21.588660 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:21.591563 (+ 2903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:21.591566 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:21.591568 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:21.591597 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:21.592918 (+ 1321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:21.593463 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:21.593466 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:21.593467 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:21.593545 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:21.593866 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:21.593869 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:21.593870 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:21.594055 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:21.594134 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:21.594766 (+ 632us) server_negotiation.cc:299] Negotiation successful
0929 10:33:21.594880 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260929 10:33:21.596014 19218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:21.582555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:41399 (local address 127.15.82.67:50271)
0929 10:33:21.583162 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:21.583220 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:21.584834 (+ 1614us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:21.585464 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:21.585471 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:21.585849 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:21.586373 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:21.586389 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:21.587278 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:21.587280 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:21.588196 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:21.588213 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:21.588715 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:21.588717 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:21.588942 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:21.589567 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:21.589583 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:21.591379 (+ 1796us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:21.593032 (+ 1653us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:21.593037 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:21.593051 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:21.593363 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:21.593624 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:21.593626 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:21.593627 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:21.593779 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:21.594142 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:21.594148 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:21.594528 (+ 380us) client_negotiation.cc:769] Sending connection context
0929 10:33:21.594697 (+ 169us) client_negotiation.cc:240] Negotiation successful
0929 10:33:21.595001 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":122,"threads_started":1}
I20260929 10:33:21.597245 19216 heartbeater.cc:347] Connected to a master server at 127.15.82.126:41399
I20260929 10:33:21.597498 19216 heartbeater.cc:464] Registering TS with master...
I20260929 10:33:21.598011 19216 heartbeater.cc:511] Master 127.15.82.126:41399 requested a full tablet report, sending...
I20260929 10:33:21.599086 18757 ts_manager.cc:194] Registered new tserver with Master: a11d7a835bb24f5b9129f13f1e5dba30 (127.15.82.67:44721)
I20260929 10:33:21.599642 18757 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.82.67@KRBTEST.COM'} at 127.15.82.67:50271
I20260929 10:33:21.609704 15689 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 29 10:33:21 dist-test-slave-vg89 krb5kdc[18712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678001, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Sep 29 10:33:21 dist-test-slave-vg89 krb5kdc[18712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678001, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:33:21.636900 18796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:21.627847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48696 (local address 127.15.82.126:41399)
0929 10:33:21.628063 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:21.628067 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:21.628140 (+ 73us) server_negotiation.cc:407] Connection header received
0929 10:33:21.628254 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:21.628257 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:21.628303 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:21.628379 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:21.629104 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:21.629639 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:21.630578 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:21.630810 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:21.630898 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:21.633448 (+ 2550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:21.633451 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:21.633453 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:21.633492 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:21.635093 (+ 1601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:21.635541 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:21.635545 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:21.635546 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:21.635650 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:21.635907 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:21.635910 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:21.635911 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:21.636167 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:21.636306 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:21.636528 (+ 222us) server_negotiation.cc:299] Negotiation successful
0929 10:33:21.636640 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
W20260929 10:33:21.637358 18757 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:48696
I20260929 10:33:21.638453 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 18819
I20260929 10:33:21.645157 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 18953
I20260929 10:33:21.650976 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 19087
I20260929 10:33:21.656646 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 18725
2026-09-29T10:33:21Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2185 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:33:21 dist-test-slave-vg89 krb5kdc[19228](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:33:21 dist-test-slave-vg89 krb5kdc[19228](info): set up 2 sockets
Sep 29 10:33:21 dist-test-slave-vg89 krb5kdc[19228](info): commencing operation
krb5kdc: starting...
W20260929 10:33:22.711237 15689 mini_kdc.cc:121] Time spent starting KDC: real 1.032s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Sep 29 10:33:22 dist-test-slave-vg89 krb5kdc[19228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678002, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:33:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:33:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.82.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab.
Entry for principal kudu/127.15.82.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab.
I20260929 10:33:22.836839 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:46607
--webserver_interface=127.15.82.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab
--principal=kudu/127.15.82.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:37601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.82.126:46607
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:33:22.946300 19241 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:33:22.946592 19241 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:33:22.946671 19241 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:33:22.950650 19241 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:33:22.950711 19241 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:33:22.950729 19241 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:33:22.950745 19241 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:33:22.950760 19241 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:33:22.956184 19241 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:37601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.82.126:46607
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab
--principal=kudu/127.15.82.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:46607
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.82.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19241
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:33:22.957295 19241 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:33:22.958367 19241 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:33:22.966981 19247 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:22.967007 19249 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:22.966981 19246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:22.967213 19241 server_base.cc:1034] running on GCE node
I20260929 10:33:22.967900 19241 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:33:22.969013 19241 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:33:22.970186 19241 hybrid_clock.cc:648] HybridClock initialized: now 1790678002970168 us; error 34 us; skew 500 ppm
Sep 29 10:33:22 dist-test-slave-vg89 krb5kdc[19228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678002, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:33:22.972997 19241 init.cc:377] Logged in from keytab as kudu/127.15.82.126@KRBTEST.COM (short username kudu)
I20260929 10:33:22.973989 19241 webserver.cc:460] Webserver started at http://127.15.82.126:37355/ using document root <none> and password file <none>
I20260929 10:33:22.974634 19241 fs_manager.cc:359] Metadata directory not provided
I20260929 10:33:22.974725 19241 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:33:22.974967 19241 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:33:22.977082 19241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/minicluster-data/master-0/data/instance:
uuid: "6cdd2cd8afd147f68ccd8b70c0271503"
format_stamp: "Formatted at 2026-09-29 10:33:22 on dist-test-slave-vg89"
server_key: "7fb602f5e1a5e3735b292f647d1dd41a"
server_key_iv: "1ac17846e7ceac2b53faa1ccdd4b9b83"
server_key_version: "encryptionkey@0"
I20260929 10:33:22.977662 19241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/minicluster-data/master-0/wal/instance:
uuid: "6cdd2cd8afd147f68ccd8b70c0271503"
format_stamp: "Formatted at 2026-09-29 10:33:22 on dist-test-slave-vg89"
server_key: "7fb602f5e1a5e3735b292f647d1dd41a"
server_key_iv: "1ac17846e7ceac2b53faa1ccdd4b9b83"
server_key_version: "encryptionkey@0"
I20260929 10:33:22.981495 19241 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260929 10:33:22.984010 19256 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:22.985260 19241 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260929 10:33:22.985402 19241 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/minicluster-data/master-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/minicluster-data/master-0/wal
uuid: "6cdd2cd8afd147f68ccd8b70c0271503"
format_stamp: "Formatted at 2026-09-29 10:33:22 on dist-test-slave-vg89"
server_key: "7fb602f5e1a5e3735b292f647d1dd41a"
server_key_iv: "1ac17846e7ceac2b53faa1ccdd4b9b83"
server_key_version: "encryptionkey@0"
I20260929 10:33:22.985519 19241 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:33:22.995561 19241 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:33:22.998348 19241 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:33:22.998581 19241 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:33:23.005856 19241 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.126:46607
I20260929 10:33:23.005878 19308 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.126:46607 every 8 connection(s)
I20260929 10:33:23.006901 19241 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
I20260929 10:33:23.009640 19309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:33:23.013427 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 19241
I20260929 10:33:23.013536 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/minicluster-data/master-0/wal/instance
I20260929 10:33:23.013837 15689 external_mini_cluster.cc:1428] Setting key 559c28dfcb8fc9597103054e5737fe30
I20260929 10:33:23.015785 19309 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6cdd2cd8afd147f68ccd8b70c0271503: Bootstrap starting.
I20260929 10:33:23.018577 19309 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6cdd2cd8afd147f68ccd8b70c0271503: Neither blocks nor log segments found. Creating new log.
I20260929 10:33:23.019475 19309 log.cc:824] T 00000000000000000000000000000000 P 6cdd2cd8afd147f68ccd8b70c0271503: Log is configured to *not* fsync() on all Append() calls
I20260929 10:33:23.021631 19309 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6cdd2cd8afd147f68ccd8b70c0271503: No bootstrap required, opened a new log
Sep 29 10:33:23 dist-test-slave-vg89 krb5kdc[19228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678002, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:33:23.026153 19309 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6cdd2cd8afd147f68ccd8b70c0271503 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cdd2cd8afd147f68ccd8b70c0271503" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 46607 } }
I20260929 10:33:23.026467 19309 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6cdd2cd8afd147f68ccd8b70c0271503 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:33:23.026532 19309 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6cdd2cd8afd147f68ccd8b70c0271503 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cdd2cd8afd147f68ccd8b70c0271503, State: Initialized, Role: FOLLOWER
I20260929 10:33:23.027007 19309 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6cdd2cd8afd147f68ccd8b70c0271503 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cdd2cd8afd147f68ccd8b70c0271503" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 46607 } }
I20260929 10:33:23.027199 19309 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6cdd2cd8afd147f68ccd8b70c0271503 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:33:23.027266 19309 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6cdd2cd8afd147f68ccd8b70c0271503 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:33:23.027379 19309 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6cdd2cd8afd147f68ccd8b70c0271503 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:33:23.028627 19309 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6cdd2cd8afd147f68ccd8b70c0271503 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cdd2cd8afd147f68ccd8b70c0271503" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 46607 } }
I20260929 10:33:23.029000 19309 leader_election.cc:302] T 00000000000000000000000000000000 P 6cdd2cd8afd147f68ccd8b70c0271503 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6cdd2cd8afd147f68ccd8b70c0271503; no voters:
I20260929 10:33:23.029332 19309 leader_election.cc:288] T 00000000000000000000000000000000 P 6cdd2cd8afd147f68ccd8b70c0271503 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:33:23.029452 19314 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6cdd2cd8afd147f68ccd8b70c0271503 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:33:23.029697 19314 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6cdd2cd8afd147f68ccd8b70c0271503 [term 1 LEADER]: Becoming Leader. State: Replica: 6cdd2cd8afd147f68ccd8b70c0271503, State: Running, Role: LEADER
I20260929 10:33:23.029971 19312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:23.015322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48886 (local address 127.15.82.126:46607)
0929 10:33:23.015764 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:23.015783 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:23.015828 (+ 45us) server_negotiation.cc:407] Connection header received
0929 10:33:23.016677 (+ 849us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:23.016701 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:23.017145 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:23.017525 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:23.018207 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:23.019778 (+ 1571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:23.020789 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:23.021473 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:23.021639 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:23.024468 (+ 2829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:23.024478 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:23.024502 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:23.024542 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:23.026582 (+ 2040us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:23.027098 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:23.027104 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:23.027110 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:23.027265 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:23.027621 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:23.027627 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:23.027632 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:23.028152 (+ 520us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:23.028372 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:23.028587 (+ 215us) server_negotiation.cc:299] Negotiation successful
0929 10:33:23.028910 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":127,"threads_started":1}
I20260929 10:33:23.030046 19314 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6cdd2cd8afd147f68ccd8b70c0271503 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cdd2cd8afd147f68ccd8b70c0271503" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 46607 } }
I20260929 10:33:23.030649 19309 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6cdd2cd8afd147f68ccd8b70c0271503 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:33:23.031316 19314 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6cdd2cd8afd147f68ccd8b70c0271503 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6cdd2cd8afd147f68ccd8b70c0271503. Latest consensus state: current_term: 1 leader_uuid: "6cdd2cd8afd147f68ccd8b70c0271503" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cdd2cd8afd147f68ccd8b70c0271503" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 46607 } } }
I20260929 10:33:23.031445 19314 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6cdd2cd8afd147f68ccd8b70c0271503 [sys.catalog]: This master's current role is: LEADER
I20260929 10:33:23.031850 19316 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6cdd2cd8afd147f68ccd8b70c0271503 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6cdd2cd8afd147f68ccd8b70c0271503" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cdd2cd8afd147f68ccd8b70c0271503" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 46607 } } }
I20260929 10:33:23.032001 19316 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6cdd2cd8afd147f68ccd8b70c0271503 [sys.catalog]: This master's current role is: LEADER
I20260929 10:33:23.032441 19323 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:33:23.035607 19323 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:33:23.041230 19323 catalog_manager.cc:1347] Generated new cluster ID: 659becc8b8e24215b834d36e6e099587
I20260929 10:33:23.041312 19323 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:33:23.056488 19323 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:33:23.057775 19323 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:33:23.063508 19323 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6cdd2cd8afd147f68ccd8b70c0271503: Generated new TSK 0
I20260929 10:33:23.064332 19323 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.82.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab.
Entry for principal kudu/127.15.82.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab.
I20260929 10:33:23.094018 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.65:0
--local_ip_for_outbound_sockets=127.15.82.65
--webserver_interface=127.15.82.65
--webserver_port=0
--tserver_master_addrs=127.15.82.126:46607
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab
--principal=kudu/127.15.82.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:37601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:33:23.206959 19335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:33:23.207290 19335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:33:23.207374 19335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:33:23.211440 19335 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:33:23.211534 19335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:33:23.211628 19335 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.65
I20260929 10:33:23.216914 19335 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:37601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab
--principal=kudu/127.15.82.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.82.65
--webserver_port=0
--tserver_master_addrs=127.15.82.126:46607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19335
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.65
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:33:23.218012 19335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:33:23.219117 19335 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:33:23.228909 19340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:23.228942 19343 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:23.228955 19341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:23.229300 19335 server_base.cc:1034] running on GCE node
I20260929 10:33:23.229627 19335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:33:23.230244 19335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:33:23.231415 19335 hybrid_clock.cc:648] HybridClock initialized: now 1790678003231400 us; error 30 us; skew 500 ppm
Sep 29 10:33:23 dist-test-slave-vg89 krb5kdc[19228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678003, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:33:23.234644 19335 init.cc:377] Logged in from keytab as kudu/127.15.82.65@KRBTEST.COM (short username kudu)
I20260929 10:33:23.235477 19335 webserver.cc:460] Webserver started at http://127.15.82.65:42669/ using document root <none> and password file <none>
I20260929 10:33:23.236037 19335 fs_manager.cc:359] Metadata directory not provided
I20260929 10:33:23.236124 19335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:33:23.236363 19335 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:33:23.238359 19335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/minicluster-data/ts-0/data/instance:
uuid: "20a5c055aed64a6c871244fd690f3859"
format_stamp: "Formatted at 2026-09-29 10:33:23 on dist-test-slave-vg89"
server_key: "470032482d81a0cc9d693c63fbfe3215"
server_key_iv: "7a486d528a535c523be2fb6dba914d18"
server_key_version: "encryptionkey@0"
I20260929 10:33:23.239006 19335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/minicluster-data/ts-0/wal/instance:
uuid: "20a5c055aed64a6c871244fd690f3859"
format_stamp: "Formatted at 2026-09-29 10:33:23 on dist-test-slave-vg89"
server_key: "470032482d81a0cc9d693c63fbfe3215"
server_key_iv: "7a486d528a535c523be2fb6dba914d18"
server_key_version: "encryptionkey@0"
I20260929 10:33:23.242748 19335 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20260929 10:33:23.245114 19350 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:23.246258 19335 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 10:33:23.246423 19335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/minicluster-data/ts-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/minicluster-data/ts-0/wal
uuid: "20a5c055aed64a6c871244fd690f3859"
format_stamp: "Formatted at 2026-09-29 10:33:23 on dist-test-slave-vg89"
server_key: "470032482d81a0cc9d693c63fbfe3215"
server_key_iv: "7a486d528a535c523be2fb6dba914d18"
server_key_version: "encryptionkey@0"
I20260929 10:33:23.246536 19335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:33:23.268968 19335 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:33:23.272101 19335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:33:23.272323 19335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:33:23.272920 19335 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:33:23.273749 19335 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:33:23.273821 19335 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:23.273893 19335 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:33:23.273947 19335 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:23.283601 19335 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.65:40863
I20260929 10:33:23.283639 19463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.65:40863 every 8 connection(s)
I20260929 10:33:23.284543 19335 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
I20260929 10:33:23.291301 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 19335
I20260929 10:33:23.291435 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/minicluster-data/ts-0/wal/instance
I20260929 10:33:23.291755 15689 external_mini_cluster.cc:1428] Setting key 6d2a186207ab8ae6b7431649d1d4183f
Sep 29 10:33:23 dist-test-slave-vg89 krb5kdc[19228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678003, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.65@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:33:23.300483 19312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:23.286504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.65:50825 (local address 127.15.82.126:46607)
0929 10:33:23.286667 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:23.286671 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:23.287498 (+ 827us) server_negotiation.cc:407] Connection header received
0929 10:33:23.289113 (+ 1615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:23.289118 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:23.289181 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:23.289277 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:23.290646 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:23.291434 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:23.292756 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:23.293148 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:23.293262 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:23.296342 (+ 3080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:23.296348 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:23.296353 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:23.296385 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:23.298106 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:23.298726 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:23.298733 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:23.298737 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:23.298834 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:23.299163 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:23.299170 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:23.299174 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:23.299448 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:23.299566 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:23.300076 (+ 510us) server_negotiation.cc:299] Negotiation successful
0929 10:33:23.300232 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260929 10:33:23.301085 19466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:23.286869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:46607 (local address 127.15.82.65:50825)
0929 10:33:23.287321 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:23.287366 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:23.288880 (+ 1514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:23.289473 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:23.289481 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:23.289874 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:23.290478 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:23.290495 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:23.291579 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:23.291589 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:23.292636 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:23.292645 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:23.293345 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:23.293350 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:23.293548 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:23.294180 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:23.294205 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:23.296151 (+ 1946us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:23.298222 (+ 2071us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:23.298227 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:23.298240 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:23.298600 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:23.298927 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:23.298930 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:23.298931 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:23.299057 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:23.299556 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:23.299561 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:23.299792 (+ 231us) client_negotiation.cc:769] Sending connection context
0929 10:33:23.299958 (+ 166us) client_negotiation.cc:240] Negotiation successful
0929 10:33:23.300210 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":101,"threads_started":1}
I20260929 10:33:23.302362 19464 heartbeater.cc:347] Connected to a master server at 127.15.82.126:46607
I20260929 10:33:23.302623 19464 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.15.82.66@KRBTEST.COM; defaulting to no policy
I20260929 10:33:23.303184 19464 heartbeater.cc:511] Master 127.15.82.126:46607 requested a full tablet report, sending...
I20260929 10:33:23.305220 19273 ts_manager.cc:194] Registered new tserver with Master: 20a5c055aed64a6c871244fd690f3859 (127.15.82.65:40863)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.66@KRBTEST.COM" created.
I20260929 10:33:23.307057 19273 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.82.65@KRBTEST.COM'} at 127.15.82.65:50825
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab.
Entry for principal kudu/127.15.82.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab.
I20260929 10:33:23.322643 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.66:0
--local_ip_for_outbound_sockets=127.15.82.66
--webserver_interface=127.15.82.66
--webserver_port=0
--tserver_master_addrs=127.15.82.126:46607
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab
--principal=kudu/127.15.82.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:37601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:33:23.435213 19469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:33:23.435497 19469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:33:23.435582 19469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:33:23.439702 19469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:33:23.439760 19469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:33:23.439884 19469 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.66
I20260929 10:33:23.445366 19469 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:37601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab
--principal=kudu/127.15.82.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.82.66
--webserver_port=0
--tserver_master_addrs=127.15.82.126:46607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19469
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.66
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:33:23.446485 19469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:33:23.447523 19469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:33:23.457448 19475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:23.457525 19477 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:23.457520 19474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:23.457639 19469 server_base.cc:1034] running on GCE node
I20260929 10:33:23.458106 19469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:33:23.458750 19469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:33:23.459944 19469 hybrid_clock.cc:648] HybridClock initialized: now 1790678003459921 us; error 34 us; skew 500 ppm
Sep 29 10:33:23 dist-test-slave-vg89 krb5kdc[19228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678003, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:33:23.462704 19469 init.cc:377] Logged in from keytab as kudu/127.15.82.66@KRBTEST.COM (short username kudu)
I20260929 10:33:23.463501 19469 webserver.cc:460] Webserver started at http://127.15.82.66:38137/ using document root <none> and password file <none>
I20260929 10:33:23.464005 19469 fs_manager.cc:359] Metadata directory not provided
I20260929 10:33:23.464061 19469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:33:23.464229 19469 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:33:23.466150 19469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/minicluster-data/ts-1/data/instance:
uuid: "e0c95de5d56342a98ca7369df8391dc4"
format_stamp: "Formatted at 2026-09-29 10:33:23 on dist-test-slave-vg89"
server_key: "3886133e3fff16df9f06c4cd82307715"
server_key_iv: "76dae794f22d9b20225bb375ce3f23e6"
server_key_version: "encryptionkey@0"
I20260929 10:33:23.466756 19469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/minicluster-data/ts-1/wal/instance:
uuid: "e0c95de5d56342a98ca7369df8391dc4"
format_stamp: "Formatted at 2026-09-29 10:33:23 on dist-test-slave-vg89"
server_key: "3886133e3fff16df9f06c4cd82307715"
server_key_iv: "76dae794f22d9b20225bb375ce3f23e6"
server_key_version: "encryptionkey@0"
I20260929 10:33:23.470369 19469 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260929 10:33:23.472849 19484 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:23.474073 19469 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 10:33:23.474223 19469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/minicluster-data/ts-1/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/minicluster-data/ts-1/wal
uuid: "e0c95de5d56342a98ca7369df8391dc4"
format_stamp: "Formatted at 2026-09-29 10:33:23 on dist-test-slave-vg89"
server_key: "3886133e3fff16df9f06c4cd82307715"
server_key_iv: "76dae794f22d9b20225bb375ce3f23e6"
server_key_version: "encryptionkey@0"
I20260929 10:33:23.474326 19469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:33:23.495960 19469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:33:23.498906 19469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:33:23.499099 19469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:33:23.499680 19469 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:33:23.500526 19469 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:33:23.500571 19469 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:23.500610 19469 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:33:23.500665 19469 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:23.509882 19469 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.66:45141
I20260929 10:33:23.509902 19597 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.66:45141 every 8 connection(s)
I20260929 10:33:23.510901 19469 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
I20260929 10:33:23.519742 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 19469
I20260929 10:33:23.519917 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/minicluster-data/ts-1/wal/instance
I20260929 10:33:23.520259 15689 external_mini_cluster.cc:1428] Setting key 12ac391415d53cf5b52ceee7a81a5d3f
Sep 29 10:33:23 dist-test-slave-vg89 krb5kdc[19228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678003, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.66@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:33:23.525729 19312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:23.512750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:44613 (local address 127.15.82.126:46607)
0929 10:33:23.512905 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:23.512910 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:23.513815 (+ 905us) server_negotiation.cc:407] Connection header received
0929 10:33:23.515313 (+ 1498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:23.515316 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:23.515367 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:23.515457 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:23.516707 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:23.517272 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:23.518526 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:23.518782 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:23.518881 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:23.521660 (+ 2779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:23.521669 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:23.521671 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:23.521696 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:23.523427 (+ 1731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:23.523946 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:23.523952 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:23.523955 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:23.524066 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:23.524523 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:23.524525 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:23.524527 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:23.524720 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:23.524819 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:23.525492 (+ 673us) server_negotiation.cc:299] Negotiation successful
0929 10:33:23.525600 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260929 10:33:23.526990 19600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:23.513208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:46607 (local address 127.15.82.66:44613)
0929 10:33:23.513659 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:23.513703 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:23.515078 (+ 1375us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:23.515630 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:23.515637 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:23.516007 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:23.516533 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:23.516547 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:23.517429 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:23.517432 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:23.518355 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:23.518365 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:23.518924 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:23.518926 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:23.519072 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:23.519772 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:23.519789 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:23.521478 (+ 1689us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:23.523559 (+ 2081us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:23.523564 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:23.523577 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:23.523839 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:23.524177 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:23.524183 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:23.524188 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:23.524416 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:23.524834 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:23.524844 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:23.525189 (+ 345us) client_negotiation.cc:769] Sending connection context
0929 10:33:23.525429 (+ 240us) client_negotiation.cc:240] Negotiation successful
0929 10:33:23.525795 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":104,"threads_started":1}
I20260929 10:33:23.528385 19598 heartbeater.cc:347] Connected to a master server at 127.15.82.126:46607
I20260929 10:33:23.528673 19598 heartbeater.cc:464] Registering TS with master...
I20260929 10:33:23.529317 19598 heartbeater.cc:511] Master 127.15.82.126:46607 requested a full tablet report, sending...
I20260929 10:33:23.530586 19273 ts_manager.cc:194] Registered new tserver with Master: e0c95de5d56342a98ca7369df8391dc4 (127.15.82.66:45141)
I20260929 10:33:23.531327 19273 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.82.66@KRBTEST.COM'} at 127.15.82.66:44613
WARNING: no policy specified for kudu/127.15.82.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab.
Entry for principal kudu/127.15.82.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab.
I20260929 10:33:23.550554 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.67:0
--local_ip_for_outbound_sockets=127.15.82.67
--webserver_interface=127.15.82.67
--webserver_port=0
--tserver_master_addrs=127.15.82.126:46607
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab
--principal=kudu/127.15.82.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:37601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:33:23.669210 19603 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:33:23.669529 19603 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:33:23.669613 19603 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:33:23.673734 19603 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:33:23.673846 19603 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:33:23.673952 19603 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.67
I20260929 10:33:23.679768 19603 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:37601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab
--principal=kudu/127.15.82.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.82.67
--webserver_port=0
--tserver_master_addrs=127.15.82.126:46607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19603
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.67
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:33:23.680977 19603 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:33:23.682045 19603 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:33:23.692210 19609 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:23.692258 19608 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:23.692261 19611 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:23.692405 19603 server_base.cc:1034] running on GCE node
I20260929 10:33:23.692996 19603 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:33:23.693706 19603 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:33:23.694888 19603 hybrid_clock.cc:648] HybridClock initialized: now 1790678003694873 us; error 27 us; skew 500 ppm
Sep 29 10:33:23 dist-test-slave-vg89 krb5kdc[19228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678003, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:33:23.697544 19603 init.cc:377] Logged in from keytab as kudu/127.15.82.67@KRBTEST.COM (short username kudu)
I20260929 10:33:23.698520 19603 webserver.cc:460] Webserver started at http://127.15.82.67:39695/ using document root <none> and password file <none>
I20260929 10:33:23.699093 19603 fs_manager.cc:359] Metadata directory not provided
I20260929 10:33:23.699183 19603 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:33:23.699424 19603 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:33:23.701507 19603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/minicluster-data/ts-2/data/instance:
uuid: "45f0d753e1604f27a3f90ece6b535546"
format_stamp: "Formatted at 2026-09-29 10:33:23 on dist-test-slave-vg89"
server_key: "e5cde819c5b52e14e125f1d3c502726e"
server_key_iv: "0f97022132351410022b6a4ffc304497"
server_key_version: "encryptionkey@0"
I20260929 10:33:23.702080 19603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/minicluster-data/ts-2/wal/instance:
uuid: "45f0d753e1604f27a3f90ece6b535546"
format_stamp: "Formatted at 2026-09-29 10:33:23 on dist-test-slave-vg89"
server_key: "e5cde819c5b52e14e125f1d3c502726e"
server_key_iv: "0f97022132351410022b6a4ffc304497"
server_key_version: "encryptionkey@0"
I20260929 10:33:23.705864 19603 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260929 10:33:23.708551 19618 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:23.709794 19603 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 10:33:23.709951 19603 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/minicluster-data/ts-2/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/minicluster-data/ts-2/wal
uuid: "45f0d753e1604f27a3f90ece6b535546"
format_stamp: "Formatted at 2026-09-29 10:33:23 on dist-test-slave-vg89"
server_key: "e5cde819c5b52e14e125f1d3c502726e"
server_key_iv: "0f97022132351410022b6a4ffc304497"
server_key_version: "encryptionkey@0"
I20260929 10:33:23.710067 19603 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:33:23.740538 19603 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:33:23.743705 19603 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:33:23.743909 19603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:33:23.744531 19603 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:33:23.745397 19603 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:33:23.745442 19603 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:23.745482 19603 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:33:23.745498 19603 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:23.754946 19603 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.67:44533
I20260929 10:33:23.754966 19731 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.67:44533 every 8 connection(s)
I20260929 10:33:23.755851 19603 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
I20260929 10:33:23.758098 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 19603
I20260929 10:33:23.758240 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790677981592896-15689-0/minicluster-data/ts-2/wal/instance
I20260929 10:33:23.758605 15689 external_mini_cluster.cc:1428] Setting key cfe7c233ef9f043ecb0fdbf9ef285844
Sep 29 10:33:23 dist-test-slave-vg89 krb5kdc[19228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678003, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.67@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:33:23.769469 19312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:23.757600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.67:44313 (local address 127.15.82.126:46607)
0929 10:33:23.757768 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:23.757773 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:23.758665 (+ 892us) server_negotiation.cc:407] Connection header received
0929 10:33:23.760279 (+ 1614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:23.760282 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:23.760325 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:23.760394 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:23.761536 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:23.762235 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:23.763514 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:23.763760 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:23.763863 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:23.766387 (+ 2524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:23.766390 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:23.766392 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:23.766459 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:23.767711 (+ 1252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:23.768187 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:23.768189 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:23.768190 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:23.768261 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:23.768555 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:23.768557 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:23.768558 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:23.768722 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:23.768799 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:23.769250 (+ 451us) server_negotiation.cc:299] Negotiation successful
0929 10:33:23.769346 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260929 10:33:23.770298 19734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:23.758008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:46607 (local address 127.15.82.67:44313)
0929 10:33:23.758517 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:23.758570 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:23.760050 (+ 1480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:23.760575 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:23.760584 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:23.760939 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:23.761412 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:23.761424 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:23.762389 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:23.762392 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:23.763344 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:23.763354 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:23.763905 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:23.763908 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:23.764067 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:23.764724 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:23.764740 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:23.766228 (+ 1488us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:23.767823 (+ 1595us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:23.767828 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:23.767840 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:23.768099 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:23.768341 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:23.768343 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:23.768344 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:23.768476 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:23.768822 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:23.768827 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:23.769053 (+ 226us) client_negotiation.cc:769] Sending connection context
0929 10:33:23.769206 (+ 153us) client_negotiation.cc:240] Negotiation successful
0929 10:33:23.769451 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":128,"threads_started":1}
I20260929 10:33:23.771525 19732 heartbeater.cc:347] Connected to a master server at 127.15.82.126:46607
I20260929 10:33:23.771754 19732 heartbeater.cc:464] Registering TS with master...
I20260929 10:33:23.772372 19732 heartbeater.cc:511] Master 127.15.82.126:46607 requested a full tablet report, sending...
I20260929 10:33:23.773360 19273 ts_manager.cc:194] Registered new tserver with Master: 45f0d753e1604f27a3f90ece6b535546 (127.15.82.67:44533)
I20260929 10:33:23.773885 19273 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.82.67@KRBTEST.COM'} at 127.15.82.67:44313
I20260929 10:33:23.784200 15689 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 29 10:33:23 dist-test-slave-vg89 krb5kdc[19228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678003, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 29 10:33:23 dist-test-slave-vg89 krb5kdc[19228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678003, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.82.65@KRBTEST.COM
I20260929 10:33:23.809675 19739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:23.800870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36084 (local address 127.15.82.65:40863)
0929 10:33:23.801198 (+ 328us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:23.801213 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:23.801238 (+ 25us) server_negotiation.cc:407] Connection header received
0929 10:33:23.801329 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:23.801334 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:23.801528 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:23.801657 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:23.802313 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:23.803125 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:23.803925 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:23.804301 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:23.804385 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:23.806695 (+ 2310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:23.806704 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:23.806717 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:23.806740 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:23.808129 (+ 1389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:23.808501 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:23.808506 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:23.808507 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:23.808580 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:23.808808 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:23.808810 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:23.808811 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:23.809029 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:23.809174 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:23.809326 (+ 152us) server_negotiation.cc:299] Negotiation successful
0929 10:33:23.809475 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":229,"thread_start_us":109,"threads_started":1}
Sep 29 10:33:23 dist-test-slave-vg89 krb5kdc[19228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678003, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:33:23.820442 19312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:23.812484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47972 (local address 127.15.82.126:46607)
0929 10:33:23.812628 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:23.812635 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:23.812652 (+ 17us) server_negotiation.cc:407] Connection header received
0929 10:33:23.812815 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:23.812818 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:23.812862 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:23.812947 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:23.813614 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:23.814115 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:23.815060 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:23.815274 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:23.815356 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:23.817627 (+ 2271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:23.817630 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:23.817632 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:23.817654 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:23.818974 (+ 1320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:23.819335 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:23.819338 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:23.819339 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:23.819408 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:23.819623 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:23.819626 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:23.819627 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:23.819835 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:23.819948 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:23.820135 (+ 187us) server_negotiation.cc:299] Negotiation successful
0929 10:33:23.820226 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
W20260929 10:33:23.820874 19273 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47972
I20260929 10:33:23.821473 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 19335
I20260929 10:33:23.827173 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 19469
I20260929 10:33:23.832778 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 19603
I20260929 10:33:23.837958 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 19241
2026-09-29T10:33:23Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2179 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:33:23 dist-test-slave-vg89 krb5kdc[19744](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:33:23 dist-test-slave-vg89 krb5kdc[19744](info): set up 2 sockets
Sep 29 10:33:23 dist-test-slave-vg89 krb5kdc[19744](info): commencing operation
krb5kdc: starting...
W20260929 10:33:24.870918 15689 mini_kdc.cc:121] Time spent starting KDC: real 1.011s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Sep 29 10:33:24 dist-test-slave-vg89 krb5kdc[19744](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678004, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:33:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:33:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.82.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab.
Entry for principal kudu/127.15.82.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab.
I20260929 10:33:24.997661 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:39689
--webserver_interface=127.15.82.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab
--principal=kudu/127.15.82.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:36313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.82.126:39689
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:33:25.111395 19757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:33:25.111747 19757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:33:25.111840 19757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:33:25.115932 19757 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:33:25.116026 19757 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:33:25.116070 19757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:33:25.116138 19757 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:33:25.116182 19757 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:33:25.121579 19757 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:36313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.82.126:39689
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab
--principal=kudu/127.15.82.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:39689
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.82.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19757
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:33:25.122794 19757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:33:25.123874 19757 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:33:25.132293 19763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:25.132280 19765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:25.132278 19762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:25.132915 19757 server_base.cc:1034] running on GCE node
I20260929 10:33:25.133467 19757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:33:25.134636 19757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:33:25.135836 19757 hybrid_clock.cc:648] HybridClock initialized: now 1790678005135824 us; error 28 us; skew 500 ppm
Sep 29 10:33:25 dist-test-slave-vg89 krb5kdc[19744](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678005, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:33:25.147035 19757 init.cc:377] Logged in from keytab as kudu/127.15.82.126@KRBTEST.COM (short username kudu)
I20260929 10:33:25.147853 19757 fs_manager.cc:359] Metadata directory not provided
I20260929 10:33:25.147951 19757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:33:25.148196 19757 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:33:25.150293 19757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/minicluster-data/master-0/data/instance:
uuid: "a1faa1af1fae4cd9860fb4b0cf366694"
format_stamp: "Formatted at 2026-09-29 10:33:25 on dist-test-slave-vg89"
server_key: "7baa7e93f11ac87c8dfdd30e8c61e52a"
server_key_iv: "5126d40badff7cc7d87e343545dadda6"
server_key_version: "encryptionkey@0"
I20260929 10:33:25.150947 19757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/minicluster-data/master-0/wal/instance:
uuid: "a1faa1af1fae4cd9860fb4b0cf366694"
format_stamp: "Formatted at 2026-09-29 10:33:25 on dist-test-slave-vg89"
server_key: "7baa7e93f11ac87c8dfdd30e8c61e52a"
server_key_iv: "5126d40badff7cc7d87e343545dadda6"
server_key_version: "encryptionkey@0"
I20260929 10:33:25.154806 19757 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260929 10:33:25.157306 19771 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:25.158620 19757 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 10:33:25.158725 19757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/minicluster-data/master-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/minicluster-data/master-0/wal
uuid: "a1faa1af1fae4cd9860fb4b0cf366694"
format_stamp: "Formatted at 2026-09-29 10:33:25 on dist-test-slave-vg89"
server_key: "7baa7e93f11ac87c8dfdd30e8c61e52a"
server_key_iv: "5126d40badff7cc7d87e343545dadda6"
server_key_version: "encryptionkey@0"
I20260929 10:33:25.158805 19757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:33:25.191043 19757 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:33:25.193818 19757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:33:25.193971 19757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:33:25.201189 19757 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.126:39689
I20260929 10:33:25.201453 19823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.126:39689 every 8 connection(s)
I20260929 10:33:25.201897 19757 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
I20260929 10:33:25.204432 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 19757
I20260929 10:33:25.204571 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/minicluster-data/master-0/wal/instance
I20260929 10:33:25.204886 15689 external_mini_cluster.cc:1428] Setting key 518054b9db30e256a7d7f924a64bcf00
I20260929 10:33:25.204905 19824 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:33:25.212078 19824 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1faa1af1fae4cd9860fb4b0cf366694: Bootstrap starting.
Sep 29 10:33:25 dist-test-slave-vg89 krb5kdc[19744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678004, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:33:25.214855 19824 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a1faa1af1fae4cd9860fb4b0cf366694: Neither blocks nor log segments found. Creating new log.
I20260929 10:33:25.215696 19824 log.cc:824] T 00000000000000000000000000000000 P a1faa1af1fae4cd9860fb4b0cf366694: Log is configured to *not* fsync() on all Append() calls
I20260929 10:33:25.217936 19824 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1faa1af1fae4cd9860fb4b0cf366694: No bootstrap required, opened a new log
I20260929 10:33:25.220290 19827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:25.206328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47376 (local address 127.15.82.126:39689)
0929 10:33:25.206788 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:25.206803 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:25.206838 (+ 35us) server_negotiation.cc:407] Connection header received
0929 10:33:25.207708 (+ 870us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:25.207725 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:25.208063 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:25.208347 (+ 284us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:25.209283 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:25.210277 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:25.211488 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:25.211948 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:25.212077 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:25.214871 (+ 2794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:25.214876 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:25.214888 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:25.214915 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:25.217051 (+ 2136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:25.217504 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:25.217510 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:25.217516 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:25.217678 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:25.217945 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:25.217951 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:25.217956 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:25.218493 (+ 537us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:25.218693 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:25.218983 (+ 290us) server_negotiation.cc:299] Negotiation successful
0929 10:33:25.219279 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":132,"threads_started":1}
I20260929 10:33:25.222468 19824 raft_consensus.cc:348] T 00000000000000000000000000000000 P a1faa1af1fae4cd9860fb4b0cf366694 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1faa1af1fae4cd9860fb4b0cf366694" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 39689 } }
I20260929 10:33:25.222709 19824 raft_consensus.cc:374] T 00000000000000000000000000000000 P a1faa1af1fae4cd9860fb4b0cf366694 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:33:25.222747 19824 raft_consensus.cc:729] T 00000000000000000000000000000000 P a1faa1af1fae4cd9860fb4b0cf366694 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1faa1af1fae4cd9860fb4b0cf366694, State: Initialized, Role: FOLLOWER
I20260929 10:33:25.223104 19824 consensus_queue.cc:260] T 00000000000000000000000000000000 P a1faa1af1fae4cd9860fb4b0cf366694 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1faa1af1fae4cd9860fb4b0cf366694" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 39689 } }
I20260929 10:33:25.223209 19824 raft_consensus.cc:388] T 00000000000000000000000000000000 P a1faa1af1fae4cd9860fb4b0cf366694 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:33:25.223254 19824 raft_consensus.cc:482] T 00000000000000000000000000000000 P a1faa1af1fae4cd9860fb4b0cf366694 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:33:25.223328 19824 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a1faa1af1fae4cd9860fb4b0cf366694 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:33:25.224303 19824 raft_consensus.cc:504] T 00000000000000000000000000000000 P a1faa1af1fae4cd9860fb4b0cf366694 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1faa1af1fae4cd9860fb4b0cf366694" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 39689 } }
I20260929 10:33:25.224644 19824 leader_election.cc:302] T 00000000000000000000000000000000 P a1faa1af1fae4cd9860fb4b0cf366694 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a1faa1af1fae4cd9860fb4b0cf366694; no voters:
I20260929 10:33:25.224941 19824 leader_election.cc:288] T 00000000000000000000000000000000 P a1faa1af1fae4cd9860fb4b0cf366694 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:33:25.225066 19829 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a1faa1af1fae4cd9860fb4b0cf366694 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:33:25.225312 19829 raft_consensus.cc:686] T 00000000000000000000000000000000 P a1faa1af1fae4cd9860fb4b0cf366694 [term 1 LEADER]: Becoming Leader. State: Replica: a1faa1af1fae4cd9860fb4b0cf366694, State: Running, Role: LEADER
I20260929 10:33:25.225597 19829 consensus_queue.cc:237] T 00000000000000000000000000000000 P a1faa1af1fae4cd9860fb4b0cf366694 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1faa1af1fae4cd9860fb4b0cf366694" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 39689 } }
I20260929 10:33:25.226099 19824 sys_catalog.cc:564] T 00000000000000000000000000000000 P a1faa1af1fae4cd9860fb4b0cf366694 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:33:25.227185 19830 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1faa1af1fae4cd9860fb4b0cf366694 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a1faa1af1fae4cd9860fb4b0cf366694" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1faa1af1fae4cd9860fb4b0cf366694" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 39689 } } }
I20260929 10:33:25.227312 19830 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1faa1af1fae4cd9860fb4b0cf366694 [sys.catalog]: This master's current role is: LEADER
I20260929 10:33:25.227628 19831 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1faa1af1fae4cd9860fb4b0cf366694 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a1faa1af1fae4cd9860fb4b0cf366694. Latest consensus state: current_term: 1 leader_uuid: "a1faa1af1fae4cd9860fb4b0cf366694" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1faa1af1fae4cd9860fb4b0cf366694" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 39689 } } }
I20260929 10:33:25.227715 19838 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:33:25.227735 19831 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1faa1af1fae4cd9860fb4b0cf366694 [sys.catalog]: This master's current role is: LEADER
I20260929 10:33:25.230705 19838 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:33:25.235894 19838 catalog_manager.cc:1347] Generated new cluster ID: f9af66b5a04a4b66b3dced8024b0d809
I20260929 10:33:25.235965 19838 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:33:25.251334 19838 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:33:25.252712 19838 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:33:25.259536 19838 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a1faa1af1fae4cd9860fb4b0cf366694: Generated new TSK 0
I20260929 10:33:25.260461 19838 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.82.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab.
Entry for principal kudu/127.15.82.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab.
I20260929 10:33:25.293211 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.65:0
--local_ip_for_outbound_sockets=127.15.82.65
--webserver_interface=127.15.82.65
--webserver_port=0
--tserver_master_addrs=127.15.82.126:39689
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab
--principal=kudu/127.15.82.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:36313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:33:25.404284 19850 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:33:25.404547 19850 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:33:25.404592 19850 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:33:25.408737 19850 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:33:25.408807 19850 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:33:25.408872 19850 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.65
I20260929 10:33:25.414198 19850 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:36313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab
--principal=kudu/127.15.82.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.82.65
--webserver_port=0
--tserver_master_addrs=127.15.82.126:39689
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19850
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.65
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:33:25.415279 19850 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:33:25.416261 19850 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:33:25.425508 19855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:25.425559 19858 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:25.425691 19850 server_base.cc:1034] running on GCE node
W20260929 10:33:25.425508 19856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:25.426326 19850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:33:25.427016 19850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:33:25.428311 19850 hybrid_clock.cc:648] HybridClock initialized: now 1790678005428292 us; error 29 us; skew 500 ppm
Sep 29 10:33:25 dist-test-slave-vg89 krb5kdc[19744](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678005, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:33:25.431188 19850 init.cc:377] Logged in from keytab as kudu/127.15.82.65@KRBTEST.COM (short username kudu)
I20260929 10:33:25.431933 19850 fs_manager.cc:359] Metadata directory not provided
I20260929 10:33:25.432025 19850 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:33:25.432261 19850 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:33:25.434266 19850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/minicluster-data/ts-0/data/instance:
uuid: "bf04219f4e774182abcab2f1546fe3aa"
format_stamp: "Formatted at 2026-09-29 10:33:25 on dist-test-slave-vg89"
server_key: "c91ab6ef98d5624f1e1220da9f67c9ad"
server_key_iv: "16053000d56754351bd519441045b87f"
server_key_version: "encryptionkey@0"
I20260929 10:33:25.434893 19850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/minicluster-data/ts-0/wal/instance:
uuid: "bf04219f4e774182abcab2f1546fe3aa"
format_stamp: "Formatted at 2026-09-29 10:33:25 on dist-test-slave-vg89"
server_key: "c91ab6ef98d5624f1e1220da9f67c9ad"
server_key_iv: "16053000d56754351bd519441045b87f"
server_key_version: "encryptionkey@0"
I20260929 10:33:25.438510 19850 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260929 10:33:25.441067 19864 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:25.442230 19850 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 10:33:25.442378 19850 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/minicluster-data/ts-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/minicluster-data/ts-0/wal
uuid: "bf04219f4e774182abcab2f1546fe3aa"
format_stamp: "Formatted at 2026-09-29 10:33:25 on dist-test-slave-vg89"
server_key: "c91ab6ef98d5624f1e1220da9f67c9ad"
server_key_iv: "16053000d56754351bd519441045b87f"
server_key_version: "encryptionkey@0"
I20260929 10:33:25.442551 19850 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:33:25.457947 19850 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:33:25.461426 19850 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:33:25.461692 19850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:33:25.462327 19850 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:33:25.463304 19850 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:33:25.463375 19850 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:25.463440 19850 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:33:25.463490 19850 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:25.472309 19850 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.65:37563
I20260929 10:33:25.472350 19977 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.65:37563 every 8 connection(s)
I20260929 10:33:25.472913 19850 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
I20260929 10:33:25.479502 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 19850
I20260929 10:33:25.479648 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/minicluster-data/ts-0/wal/instance
I20260929 10:33:25.479943 15689 external_mini_cluster.cc:1428] Setting key e3309cc5b2ff486534380af0b54de387
Sep 29 10:33:25 dist-test-slave-vg89 krb5kdc[19744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678005, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.65@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:33:25.487519 19827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:25.474835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.65:37887 (local address 127.15.82.126:39689)
0929 10:33:25.474969 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:25.474972 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:25.476114 (+ 1142us) server_negotiation.cc:407] Connection header received
0929 10:33:25.476931 (+ 817us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:25.476934 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:25.476980 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:25.477052 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:25.478572 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:25.479138 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:25.480529 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:25.480820 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:25.480907 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:25.483665 (+ 2758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:25.483668 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:25.483670 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:25.483693 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:25.485167 (+ 1474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:25.485839 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:25.485842 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:25.485843 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:25.485919 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:25.486313 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:25.486315 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:25.486316 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:25.486542 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:25.486618 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:25.487297 (+ 679us) server_negotiation.cc:299] Negotiation successful
0929 10:33:25.487402 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 10:33:25.488838 19980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:25.475206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:39689 (local address 127.15.82.65:37887)
0929 10:33:25.475970 (+ 764us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:25.476010 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:25.476753 (+ 743us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:25.477183 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:25.477189 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:25.477549 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:25.478357 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:25.478400 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:25.479286 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:25.479290 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:25.480405 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:25.480416 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:25.480975 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:25.480978 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:25.481178 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:25.481848 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:25.481864 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:25.483477 (+ 1613us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:25.485292 (+ 1815us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:25.485301 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:25.485320 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:25.485722 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:25.486017 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:25.486022 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:25.486026 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:25.486212 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:25.486654 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:25.486662 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:25.487023 (+ 361us) client_negotiation.cc:769] Sending connection context
0929 10:33:25.487283 (+ 260us) client_negotiation.cc:240] Negotiation successful
0929 10:33:25.487687 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":104,"threads_started":1}
I20260929 10:33:25.490129 19978 heartbeater.cc:347] Connected to a master server at 127.15.82.126:39689
WARNING: no policy specified for kudu/127.15.82.66@KRBTEST.COM; defaulting to no policy
I20260929 10:33:25.490423 19978 heartbeater.cc:464] Registering TS with master...
I20260929 10:33:25.490955 19978 heartbeater.cc:511] Master 127.15.82.126:39689 requested a full tablet report, sending...
I20260929 10:33:25.492569 19788 ts_manager.cc:194] Registered new tserver with Master: bf04219f4e774182abcab2f1546fe3aa (127.15.82.65:37563)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.66@KRBTEST.COM" created.
I20260929 10:33:25.493893 19788 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.82.65@KRBTEST.COM'} at 127.15.82.65:37887
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab.
Entry for principal kudu/127.15.82.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab.
I20260929 10:33:25.507625 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.66:0
--local_ip_for_outbound_sockets=127.15.82.66
--webserver_interface=127.15.82.66
--webserver_port=0
--tserver_master_addrs=127.15.82.126:39689
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab
--principal=kudu/127.15.82.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:36313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:33:25.625123 19983 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:33:25.625371 19983 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:33:25.625461 19983 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:33:25.629827 19983 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:33:25.629884 19983 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:33:25.630000 19983 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.66
I20260929 10:33:25.635212 19983 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:36313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab
--principal=kudu/127.15.82.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.82.66
--webserver_port=0
--tserver_master_addrs=127.15.82.126:39689
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19983
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.66
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:33:25.636308 19983 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:33:25.637383 19983 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:33:25.647800 19989 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:25.647799 19991 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:25.647795 19988 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:25.648146 19983 server_base.cc:1034] running on GCE node
I20260929 10:33:25.648548 19983 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:33:25.649267 19983 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:33:25.650480 19983 hybrid_clock.cc:648] HybridClock initialized: now 1790678005650457 us; error 32 us; skew 500 ppm
Sep 29 10:33:25 dist-test-slave-vg89 krb5kdc[19744](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678005, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:33:25.653416 19983 init.cc:377] Logged in from keytab as kudu/127.15.82.66@KRBTEST.COM (short username kudu)
I20260929 10:33:25.654250 19983 fs_manager.cc:359] Metadata directory not provided
I20260929 10:33:25.654315 19983 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:33:25.654625 19983 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:33:25.656771 19983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/minicluster-data/ts-1/data/instance:
uuid: "b1dfc55b1eb94835b2dc704f702bd04d"
format_stamp: "Formatted at 2026-09-29 10:33:25 on dist-test-slave-vg89"
server_key: "71c40447a97637001a3a741881b57165"
server_key_iv: "1a9942bf89838c1b50e9162d7094667d"
server_key_version: "encryptionkey@0"
I20260929 10:33:25.657346 19983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/minicluster-data/ts-1/wal/instance:
uuid: "b1dfc55b1eb94835b2dc704f702bd04d"
format_stamp: "Formatted at 2026-09-29 10:33:25 on dist-test-slave-vg89"
server_key: "71c40447a97637001a3a741881b57165"
server_key_iv: "1a9942bf89838c1b50e9162d7094667d"
server_key_version: "encryptionkey@0"
I20260929 10:33:25.661166 19983 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260929 10:33:25.663853 19997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:25.665126 19983 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260929 10:33:25.665272 19983 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/minicluster-data/ts-1/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/minicluster-data/ts-1/wal
uuid: "b1dfc55b1eb94835b2dc704f702bd04d"
format_stamp: "Formatted at 2026-09-29 10:33:25 on dist-test-slave-vg89"
server_key: "71c40447a97637001a3a741881b57165"
server_key_iv: "1a9942bf89838c1b50e9162d7094667d"
server_key_version: "encryptionkey@0"
I20260929 10:33:25.665395 19983 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:33:25.688916 19983 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:33:25.691987 19983 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:33:25.692193 19983 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:33:25.692785 19983 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:33:25.693595 19983 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:33:25.693665 19983 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:25.693737 19983 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:33:25.693785 19983 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:25.703349 19983 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.66:33317
I20260929 10:33:25.703365 20110 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.66:33317 every 8 connection(s)
I20260929 10:33:25.703946 19983 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
Sep 29 10:33:25 dist-test-slave-vg89 krb5kdc[19744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678005, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.66@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:33:25.714268 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 19983
I20260929 10:33:25.714478 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/minicluster-data/ts-1/wal/instance
I20260929 10:33:25.714892 15689 external_mini_cluster.cc:1428] Setting key 5bee2e6d835c1d2a30105e32ab9f5b4f
I20260929 10:33:25.719333 19827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:25.705655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:45059 (local address 127.15.82.126:39689)
0929 10:33:25.705830 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:25.705835 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:25.706717 (+ 882us) server_negotiation.cc:407] Connection header received
0929 10:33:25.707987 (+ 1270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:25.707990 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:25.708038 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:25.708116 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:25.709294 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:25.709836 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:25.711011 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:25.711231 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:25.711316 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:25.713784 (+ 2468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:25.713787 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:25.713789 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:25.713815 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:25.715477 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:25.716074 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:25.716077 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:25.716079 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:25.716182 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:25.716810 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:25.716813 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:25.716815 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:25.717106 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:25.717210 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:25.718953 (+ 1743us) server_negotiation.cc:299] Negotiation successful
0929 10:33:25.719112 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260929 10:33:25.720710 20113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:25.706067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:39689 (local address 127.15.82.66:45059)
0929 10:33:25.706565 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:25.706608 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:25.707789 (+ 1181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:25.708284 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:25.708292 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:25.708648 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:25.709154 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:25.709168 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:25.709945 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:25.709947 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:25.710908 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:25.710917 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:25.711367 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:25.711369 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:25.711506 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:25.712135 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:25.712152 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:25.713623 (+ 1471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:25.715593 (+ 1970us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:25.715599 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:25.715615 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:25.715976 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:25.716442 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:25.716445 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:25.716446 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:25.716681 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:25.718211 (+ 1530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:25.718219 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:25.718586 (+ 367us) client_negotiation.cc:769] Sending connection context
0929 10:33:25.718871 (+ 285us) client_negotiation.cc:240] Negotiation successful
0929 10:33:25.719254 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":106,"threads_started":1}
I20260929 10:33:25.722857 20111 heartbeater.cc:347] Connected to a master server at 127.15.82.126:39689
I20260929 10:33:25.723135 20111 heartbeater.cc:464] Registering TS with master...
I20260929 10:33:25.723733 20111 heartbeater.cc:511] Master 127.15.82.126:39689 requested a full tablet report, sending...
I20260929 10:33:25.725229 19788 ts_manager.cc:194] Registered new tserver with Master: b1dfc55b1eb94835b2dc704f702bd04d (127.15.82.66:33317)
I20260929 10:33:25.726114 19788 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.82.66@KRBTEST.COM'} at 127.15.82.66:45059
WARNING: no policy specified for kudu/127.15.82.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab.
Entry for principal kudu/127.15.82.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab.
I20260929 10:33:25.746572 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.67:0
--local_ip_for_outbound_sockets=127.15.82.67
--webserver_interface=127.15.82.67
--webserver_port=0
--tserver_master_addrs=127.15.82.126:39689
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab
--principal=kudu/127.15.82.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:36313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:33:25.857260 20116 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:33:25.857497 20116 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:33:25.857538 20116 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:33:25.861653 20116 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:33:25.861706 20116 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:33:25.861768 20116 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.67
I20260929 10:33:25.867249 20116 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:36313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab
--principal=kudu/127.15.82.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.82.67
--webserver_port=0
--tserver_master_addrs=127.15.82.126:39689
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20116
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.67
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:33:25.868146 20116 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:33:25.869123 20116 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:33:25.878389 20121 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:25.878468 20124 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:25.878530 20116 server_base.cc:1034] running on GCE node
W20260929 10:33:25.878638 20122 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:25.879040 20116 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:33:25.879638 20116 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:33:25.880821 20116 hybrid_clock.cc:648] HybridClock initialized: now 1790678005880806 us; error 25 us; skew 500 ppm
Sep 29 10:33:25 dist-test-slave-vg89 krb5kdc[19744](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678005, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:33:25.883620 20116 init.cc:377] Logged in from keytab as kudu/127.15.82.67@KRBTEST.COM (short username kudu)
I20260929 10:33:25.884325 20116 fs_manager.cc:359] Metadata directory not provided
I20260929 10:33:25.884424 20116 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:33:25.884675 20116 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:33:25.886849 20116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/minicluster-data/ts-2/data/instance:
uuid: "8f196d39207e4dc4aa3af37234966c83"
format_stamp: "Formatted at 2026-09-29 10:33:25 on dist-test-slave-vg89"
server_key: "19e053c4348861d3e6a9b7ae035d11fc"
server_key_iv: "610f6e37f5b66d926a5ae4c8f6fbf29f"
server_key_version: "encryptionkey@0"
I20260929 10:33:25.887391 20116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/minicluster-data/ts-2/wal/instance:
uuid: "8f196d39207e4dc4aa3af37234966c83"
format_stamp: "Formatted at 2026-09-29 10:33:25 on dist-test-slave-vg89"
server_key: "19e053c4348861d3e6a9b7ae035d11fc"
server_key_iv: "610f6e37f5b66d926a5ae4c8f6fbf29f"
server_key_version: "encryptionkey@0"
I20260929 10:33:25.891055 20116 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260929 10:33:25.893347 20130 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:25.894513 20116 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 10:33:25.894641 20116 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/minicluster-data/ts-2/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/minicluster-data/ts-2/wal
uuid: "8f196d39207e4dc4aa3af37234966c83"
format_stamp: "Formatted at 2026-09-29 10:33:25 on dist-test-slave-vg89"
server_key: "19e053c4348861d3e6a9b7ae035d11fc"
server_key_iv: "610f6e37f5b66d926a5ae4c8f6fbf29f"
server_key_version: "encryptionkey@0"
I20260929 10:33:25.894743 20116 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:33:25.931149 20116 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:33:25.934616 20116 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:33:25.934821 20116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:33:25.935413 20116 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:33:25.936254 20116 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:33:25.936326 20116 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:25.936399 20116 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:33:25.936442 20116 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:25.945149 20116 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.67:38487
I20260929 10:33:25.945171 20243 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.67:38487 every 8 connection(s)
I20260929 10:33:25.945747 20116 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
I20260929 10:33:25.953408 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 20116
I20260929 10:33:25.953549 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/minicluster-data/ts-2/wal/instance
I20260929 10:33:25.953838 15689 external_mini_cluster.cc:1428] Setting key 33ca79ee1ea24bf9cc839d8429773bd6
Sep 29 10:33:25 dist-test-slave-vg89 krb5kdc[19744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678005, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.67@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:33:25.959450 19827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:25.947701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.67:48437 (local address 127.15.82.126:39689)
0929 10:33:25.947855 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:25.947860 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:25.949063 (+ 1203us) server_negotiation.cc:407] Connection header received
0929 10:33:25.949871 (+ 808us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:25.949874 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:25.949920 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:25.949995 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:25.951214 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:25.951748 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:25.952810 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:25.953017 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:25.953092 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:25.955858 (+ 2766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:25.955861 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:25.955864 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:25.955889 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:25.957380 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:25.957899 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:25.957903 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:25.957904 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:25.957990 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:25.958317 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:25.958322 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:25.958324 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:25.958567 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:25.958666 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:25.959155 (+ 489us) server_negotiation.cc:299] Negotiation successful
0929 10:33:25.959282 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260929 10:33:25.960207 20246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:25.948151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:39689 (local address 127.15.82.67:48437)
0929 10:33:25.948916 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:25.948957 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:25.949695 (+ 738us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:25.950159 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:25.950166 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:25.950618 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:25.951092 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:25.951104 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:25.951854 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:25.951856 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:25.952712 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:25.952720 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:25.953137 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:25.953140 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:25.953303 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:25.953967 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:25.953988 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:25.955687 (+ 1699us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:25.957513 (+ 1826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:25.957519 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:25.957533 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:25.957792 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:25.958078 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:25.958082 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:25.958084 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:25.958223 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:25.958675 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:25.958683 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:25.958923 (+ 240us) client_negotiation.cc:769] Sending connection context
0929 10:33:25.959101 (+ 178us) client_negotiation.cc:240] Negotiation successful
0929 10:33:25.959408 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":109,"threads_started":1}
I20260929 10:33:25.961233 20244 heartbeater.cc:347] Connected to a master server at 127.15.82.126:39689
I20260929 10:33:25.961489 20244 heartbeater.cc:464] Registering TS with master...
I20260929 10:33:25.962013 20244 heartbeater.cc:511] Master 127.15.82.126:39689 requested a full tablet report, sending...
I20260929 10:33:25.963047 19788 ts_manager.cc:194] Registered new tserver with Master: 8f196d39207e4dc4aa3af37234966c83 (127.15.82.67:38487)
I20260929 10:33:25.963554 19788 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.82.67@KRBTEST.COM'} at 127.15.82.67:48437
I20260929 10:33:25.967702 15689 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 10:33:25.977288 19827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:25.969646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47380 (local address 127.15.82.126:39689)
0929 10:33:25.969807 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:25.969811 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:25.970017 (+ 206us) server_negotiation.cc:407] Connection header received
0929 10:33:25.970150 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:25.970153 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:25.970200 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:25.970286 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:25.971008 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:25.971519 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:25.972356 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:25.972587 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:25.972705 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:25.974103 (+ 1398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:25.974106 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:25.974109 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:25.974132 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:25.975631 (+ 1499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:25.976056 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:25.976061 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:25.976063 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:25.976143 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:25.976425 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:25.976429 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:25.976431 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:25.976636 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:25.976782 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:25.977006 (+ 224us) server_negotiation.cc:299] Negotiation successful
0929 10:33:25.977119 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 10:33:25.980310 19788 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47380:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 10:33:25.983407 19788 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:33:25.997747 20257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:25.992528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.66:33317 (local address 127.0.0.1:45346)
0929 10:33:25.993241 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:25.993255 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:25.993362 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:25.993848 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:25.993851 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:25.993868 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:33:25.993976 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:25.993983 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:25.995197 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:25.995201 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:25.996378 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:25.996386 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:25.997164 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:25.997168 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:25.997283 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:25.997322 (+ 39us) client_negotiation.cc:769] Sending connection context
0929 10:33:25.997460 (+ 138us) client_negotiation.cc:240] Negotiation successful
0929 10:33:25.997582 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":651,"mutex_wait_us":48,"thread_start_us":107,"threads_started":1}
I20260929 10:33:25.997884 20256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:25.991856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37388 (local address 127.15.82.65:37563)
0929 10:33:25.992183 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:25.992190 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:25.992339 (+ 149us) server_negotiation.cc:407] Connection header received
0929 10:33:25.992633 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:25.992639 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:25.992883 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:25.993042 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:33:25.993353 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:25.994512 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:25.996249 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:25.997046 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:25.997153 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:25.997308 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:25.997632 (+ 324us) server_negotiation.cc:299] Negotiation successful
0929 10:33:25.997729 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":107,"threads_started":1}
I20260929 10:33:25.997881 20258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:25.992692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45346 (local address 127.15.82.66:33317)
0929 10:33:25.993104 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:25.993117 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:25.993466 (+ 349us) server_negotiation.cc:407] Connection header received
0929 10:33:25.993547 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:25.993551 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:25.993713 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:25.993873 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:33:25.994081 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:25.995049 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:25.996491 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:25.997035 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:25.997160 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:25.997305 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:25.997632 (+ 327us) server_negotiation.cc:299] Negotiation successful
0929 10:33:25.997729 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":93,"threads_started":1}
I20260929 10:33:25.998183 20260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:25.993268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48042 (local address 127.15.82.67:38487)
0929 10:33:25.993635 (+ 367us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:25.993647 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:25.993668 (+ 21us) server_negotiation.cc:407] Connection header received
0929 10:33:25.993750 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:25.993755 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:25.993931 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:25.994313 (+ 382us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:33:25.994473 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:25.995410 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:25.996576 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:25.997322 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:25.997426 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:25.997549 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:25.997913 (+ 364us) server_negotiation.cc:299] Negotiation successful
0929 10:33:25.998009 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":157,"threads_started":1}
I20260929 10:33:25.997798 20259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:25.992869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.67:38487 (local address 127.0.0.1:48042)
0929 10:33:25.993300 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:25.993316 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:25.993395 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:25.994079 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:25.994082 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:25.994099 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:33:25.994186 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:25.994191 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:25.995551 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:25.995554 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:25.996431 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:25.996441 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:25.997441 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:25.997448 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:25.997532 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:25.997551 (+ 19us) client_negotiation.cc:769] Sending connection context
0929 10:33:25.997633 (+ 82us) client_negotiation.cc:240] Negotiation successful
0929 10:33:25.997689 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":83,"threads_started":1}
I20260929 10:33:25.998358 20255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:25.991740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.65:37563 (local address 127.0.0.1:37388)
0929 10:33:25.992202 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:25.992257 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:25.992424 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:25.993035 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:25.993041 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:25.993063 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:33:25.993203 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:25.993210 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:25.994627 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:25.994630 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:25.996115 (+ 1485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:25.996126 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:25.997377 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:25.997383 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:25.997472 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:25.997494 (+ 22us) client_negotiation.cc:769] Sending connection context
0929 10:33:25.998135 (+ 641us) client_negotiation.cc:240] Negotiation successful
0929 10:33:25.998213 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"mutex_wait_us":20,"thread_start_us":87,"threads_started":1}
I20260929 10:33:26.001186 20178 tablet_service.cc:1467] Processing CreateTablet for tablet b158daabef7342ff9769bc5ed2f20f44 (DEFAULT_TABLE table=test-table [id=83c79fb51dcc4320a0e97c2ffe0958ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:33:26.001340 19912 tablet_service.cc:1467] Processing CreateTablet for tablet b158daabef7342ff9769bc5ed2f20f44 (DEFAULT_TABLE table=test-table [id=83c79fb51dcc4320a0e97c2ffe0958ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:33:26.001343 20043 tablet_service.cc:1467] Processing CreateTablet for tablet b158daabef7342ff9769bc5ed2f20f44 (DEFAULT_TABLE table=test-table [id=83c79fb51dcc4320a0e97c2ffe0958ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:33:26.002292 20178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b158daabef7342ff9769bc5ed2f20f44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:33:26.002357 19912 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b158daabef7342ff9769bc5ed2f20f44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:33:26.002357 20043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b158daabef7342ff9769bc5ed2f20f44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:33:26.008430 20261 tablet_bootstrap.cc:492] T b158daabef7342ff9769bc5ed2f20f44 P 8f196d39207e4dc4aa3af37234966c83: Bootstrap starting.
I20260929 10:33:26.009968 20263 tablet_bootstrap.cc:492] T b158daabef7342ff9769bc5ed2f20f44 P b1dfc55b1eb94835b2dc704f702bd04d: Bootstrap starting.
I20260929 10:33:26.010727 20261 tablet_bootstrap.cc:654] T b158daabef7342ff9769bc5ed2f20f44 P 8f196d39207e4dc4aa3af37234966c83: Neither blocks nor log segments found. Creating new log.
I20260929 10:33:26.010725 20262 tablet_bootstrap.cc:492] T b158daabef7342ff9769bc5ed2f20f44 P bf04219f4e774182abcab2f1546fe3aa: Bootstrap starting.
I20260929 10:33:26.011772 20261 log.cc:824] T b158daabef7342ff9769bc5ed2f20f44 P 8f196d39207e4dc4aa3af37234966c83: Log is configured to *not* fsync() on all Append() calls
I20260929 10:33:26.012117 20263 tablet_bootstrap.cc:654] T b158daabef7342ff9769bc5ed2f20f44 P b1dfc55b1eb94835b2dc704f702bd04d: Neither blocks nor log segments found. Creating new log.
I20260929 10:33:26.012874 20262 tablet_bootstrap.cc:654] T b158daabef7342ff9769bc5ed2f20f44 P bf04219f4e774182abcab2f1546fe3aa: Neither blocks nor log segments found. Creating new log.
I20260929 10:33:26.013211 20263 log.cc:824] T b158daabef7342ff9769bc5ed2f20f44 P b1dfc55b1eb94835b2dc704f702bd04d: Log is configured to *not* fsync() on all Append() calls
I20260929 10:33:26.013643 20262 log.cc:824] T b158daabef7342ff9769bc5ed2f20f44 P bf04219f4e774182abcab2f1546fe3aa: Log is configured to *not* fsync() on all Append() calls
I20260929 10:33:26.014117 20261 tablet_bootstrap.cc:492] T b158daabef7342ff9769bc5ed2f20f44 P 8f196d39207e4dc4aa3af37234966c83: No bootstrap required, opened a new log
I20260929 10:33:26.014333 20261 ts_tablet_manager.cc:1397] T b158daabef7342ff9769bc5ed2f20f44 P 8f196d39207e4dc4aa3af37234966c83: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260929 10:33:26.015183 20263 tablet_bootstrap.cc:492] T b158daabef7342ff9769bc5ed2f20f44 P b1dfc55b1eb94835b2dc704f702bd04d: No bootstrap required, opened a new log
I20260929 10:33:26.015381 20263 ts_tablet_manager.cc:1397] T b158daabef7342ff9769bc5ed2f20f44 P b1dfc55b1eb94835b2dc704f702bd04d: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260929 10:33:26.015508 20262 tablet_bootstrap.cc:492] T b158daabef7342ff9769bc5ed2f20f44 P bf04219f4e774182abcab2f1546fe3aa: No bootstrap required, opened a new log
I20260929 10:33:26.015674 20262 ts_tablet_manager.cc:1397] T b158daabef7342ff9769bc5ed2f20f44 P bf04219f4e774182abcab2f1546fe3aa: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260929 10:33:26.017382 20261 raft_consensus.cc:348] T b158daabef7342ff9769bc5ed2f20f44 P 8f196d39207e4dc4aa3af37234966c83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf04219f4e774182abcab2f1546fe3aa" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 37563 } } peers { permanent_uuid: "b1dfc55b1eb94835b2dc704f702bd04d" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 33317 } } peers { permanent_uuid: "8f196d39207e4dc4aa3af37234966c83" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 38487 } }
I20260929 10:33:26.017630 20261 raft_consensus.cc:374] T b158daabef7342ff9769bc5ed2f20f44 P 8f196d39207e4dc4aa3af37234966c83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:33:26.017684 20261 raft_consensus.cc:729] T b158daabef7342ff9769bc5ed2f20f44 P 8f196d39207e4dc4aa3af37234966c83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f196d39207e4dc4aa3af37234966c83, State: Initialized, Role: FOLLOWER
I20260929 10:33:26.018146 20261 consensus_queue.cc:260] T b158daabef7342ff9769bc5ed2f20f44 P 8f196d39207e4dc4aa3af37234966c83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf04219f4e774182abcab2f1546fe3aa" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 37563 } } peers { permanent_uuid: "b1dfc55b1eb94835b2dc704f702bd04d" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 33317 } } peers { permanent_uuid: "8f196d39207e4dc4aa3af37234966c83" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 38487 } }
I20260929 10:33:26.018776 20244 heartbeater.cc:503] Master 127.15.82.126:39689 was elected leader, sending a full tablet report...
I20260929 10:33:26.019361 20263 raft_consensus.cc:348] T b158daabef7342ff9769bc5ed2f20f44 P b1dfc55b1eb94835b2dc704f702bd04d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf04219f4e774182abcab2f1546fe3aa" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 37563 } } peers { permanent_uuid: "b1dfc55b1eb94835b2dc704f702bd04d" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 33317 } } peers { permanent_uuid: "8f196d39207e4dc4aa3af37234966c83" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 38487 } }
I20260929 10:33:26.019615 20263 raft_consensus.cc:374] T b158daabef7342ff9769bc5ed2f20f44 P b1dfc55b1eb94835b2dc704f702bd04d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:33:26.019665 20263 raft_consensus.cc:729] T b158daabef7342ff9769bc5ed2f20f44 P b1dfc55b1eb94835b2dc704f702bd04d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1dfc55b1eb94835b2dc704f702bd04d, State: Initialized, Role: FOLLOWER
I20260929 10:33:26.020004 20262 raft_consensus.cc:348] T b158daabef7342ff9769bc5ed2f20f44 P bf04219f4e774182abcab2f1546fe3aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf04219f4e774182abcab2f1546fe3aa" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 37563 } } peers { permanent_uuid: "b1dfc55b1eb94835b2dc704f702bd04d" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 33317 } } peers { permanent_uuid: "8f196d39207e4dc4aa3af37234966c83" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 38487 } }
I20260929 10:33:26.020115 20263 consensus_queue.cc:260] T b158daabef7342ff9769bc5ed2f20f44 P b1dfc55b1eb94835b2dc704f702bd04d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf04219f4e774182abcab2f1546fe3aa" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 37563 } } peers { permanent_uuid: "b1dfc55b1eb94835b2dc704f702bd04d" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 33317 } } peers { permanent_uuid: "8f196d39207e4dc4aa3af37234966c83" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 38487 } }
I20260929 10:33:26.020232 20262 raft_consensus.cc:374] T b158daabef7342ff9769bc5ed2f20f44 P bf04219f4e774182abcab2f1546fe3aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:33:26.020288 20262 raft_consensus.cc:729] T b158daabef7342ff9769bc5ed2f20f44 P bf04219f4e774182abcab2f1546fe3aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf04219f4e774182abcab2f1546fe3aa, State: Initialized, Role: FOLLOWER
I20260929 10:33:26.020690 20261 ts_tablet_manager.cc:1428] T b158daabef7342ff9769bc5ed2f20f44 P 8f196d39207e4dc4aa3af37234966c83: Time spent starting tablet: real 0.006s user 0.001s sys 0.003s
I20260929 10:33:26.020712 20262 consensus_queue.cc:260] T b158daabef7342ff9769bc5ed2f20f44 P bf04219f4e774182abcab2f1546fe3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf04219f4e774182abcab2f1546fe3aa" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 37563 } } peers { permanent_uuid: "b1dfc55b1eb94835b2dc704f702bd04d" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 33317 } } peers { permanent_uuid: "8f196d39207e4dc4aa3af37234966c83" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 38487 } }
I20260929 10:33:26.021077 20263 ts_tablet_manager.cc:1428] T b158daabef7342ff9769bc5ed2f20f44 P b1dfc55b1eb94835b2dc704f702bd04d: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260929 10:33:26.021407 20111 heartbeater.cc:503] Master 127.15.82.126:39689 was elected leader, sending a full tablet report...
I20260929 10:33:26.021448 19978 heartbeater.cc:503] Master 127.15.82.126:39689 was elected leader, sending a full tablet report...
I20260929 10:33:26.021646 20262 ts_tablet_manager.cc:1428] T b158daabef7342ff9769bc5ed2f20f44 P bf04219f4e774182abcab2f1546fe3aa: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260929 10:33:26.137759 20267 raft_consensus.cc:482] T b158daabef7342ff9769bc5ed2f20f44 P 8f196d39207e4dc4aa3af37234966c83 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:33:26.137965 20267 raft_consensus.cc:504] T b158daabef7342ff9769bc5ed2f20f44 P 8f196d39207e4dc4aa3af37234966c83 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf04219f4e774182abcab2f1546fe3aa" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 37563 } } peers { permanent_uuid: "b1dfc55b1eb94835b2dc704f702bd04d" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 33317 } } peers { permanent_uuid: "8f196d39207e4dc4aa3af37234966c83" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 38487 } }
I20260929 10:33:26.139140 20267 leader_election.cc:288] T b158daabef7342ff9769bc5ed2f20f44 P 8f196d39207e4dc4aa3af37234966c83 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bf04219f4e774182abcab2f1546fe3aa (127.15.82.65:37563), b1dfc55b1eb94835b2dc704f702bd04d (127.15.82.66:33317)
I20260929 10:33:26.143975 20246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:26.139410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.65:37563 (local address 127.15.82.67:53427)
0929 10:33:26.139562 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:26.139594 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:26.139708 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:26.140086 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:26.140090 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:26.140109 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:33:26.140263 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:26.140272 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:26.141181 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:26.141185 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:26.142210 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:26.142224 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:26.143411 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:26.143417 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:26.143561 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:26.143600 (+ 39us) client_negotiation.cc:769] Sending connection context
0929 10:33:26.143680 (+ 80us) client_negotiation.cc:240] Negotiation successful
0929 10:33:26.143774 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260929 10:33:26.144281 20256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:26.139541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.67:53427 (local address 127.15.82.65:37563)
0929 10:33:26.139681 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:26.139685 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:26.139702 (+ 17us) server_negotiation.cc:407] Connection header received
0929 10:33:26.139870 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:26.139876 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:26.139936 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:26.140036 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:33:26.140362 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:26.141069 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:26.142371 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:26.143103 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:26.143266 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:26.143329 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:26.144032 (+ 703us) server_negotiation.cc:299] Negotiation successful
0929 10:33:26.144124 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260929 10:33:26.144330 20258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:26.139879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.67:55753 (local address 127.15.82.66:33317)
0929 10:33:26.140017 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:26.140021 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:26.140231 (+ 210us) server_negotiation.cc:407] Connection header received
0929 10:33:26.140343 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:26.140346 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:26.140406 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:26.140494 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:33:26.140772 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:26.141415 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:26.143002 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:26.143686 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:26.143822 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:26.143871 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:26.144080 (+ 209us) server_negotiation.cc:299] Negotiation successful
0929 10:33:26.144177 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260929 10:33:26.144754 20270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:26.139748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.66:33317 (local address 127.15.82.67:55753)
0929 10:33:26.140130 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:26.140142 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:26.140235 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:26.140508 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:26.140511 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:26.140528 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:33:26.140655 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:26.140662 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:26.141529 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:26.141534 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:26.142861 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:26.142874 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:26.143823 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:26.143826 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:26.143927 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:26.143950 (+ 23us) client_negotiation.cc:769] Sending connection context
0929 10:33:26.144494 (+ 544us) client_negotiation.cc:240] Negotiation successful
0929 10:33:26.144597 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":127,"threads_started":1}
I20260929 10:33:26.145457 20065 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b158daabef7342ff9769bc5ed2f20f44" candidate_uuid: "8f196d39207e4dc4aa3af37234966c83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1dfc55b1eb94835b2dc704f702bd04d" is_pre_election: true
I20260929 10:33:26.145539 19932 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b158daabef7342ff9769bc5ed2f20f44" candidate_uuid: "8f196d39207e4dc4aa3af37234966c83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf04219f4e774182abcab2f1546fe3aa" is_pre_election: true
I20260929 10:33:26.145836 20065 raft_consensus.cc:2454] T b158daabef7342ff9769bc5ed2f20f44 P b1dfc55b1eb94835b2dc704f702bd04d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f196d39207e4dc4aa3af37234966c83 in term 0.
I20260929 10:33:26.145880 19932 raft_consensus.cc:2454] T b158daabef7342ff9769bc5ed2f20f44 P bf04219f4e774182abcab2f1546fe3aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f196d39207e4dc4aa3af37234966c83 in term 0.
I20260929 10:33:26.146443 20133 leader_election.cc:302] T b158daabef7342ff9769bc5ed2f20f44 P 8f196d39207e4dc4aa3af37234966c83 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f196d39207e4dc4aa3af37234966c83, bf04219f4e774182abcab2f1546fe3aa; no voters:
I20260929 10:33:26.146767 20267 raft_consensus.cc:2781] T b158daabef7342ff9769bc5ed2f20f44 P 8f196d39207e4dc4aa3af37234966c83 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:33:26.146868 20267 raft_consensus.cc:482] T b158daabef7342ff9769bc5ed2f20f44 P 8f196d39207e4dc4aa3af37234966c83 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:33:26.146931 20267 raft_consensus.cc:3037] T b158daabef7342ff9769bc5ed2f20f44 P 8f196d39207e4dc4aa3af37234966c83 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:33:26.148236 20267 raft_consensus.cc:504] T b158daabef7342ff9769bc5ed2f20f44 P 8f196d39207e4dc4aa3af37234966c83 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf04219f4e774182abcab2f1546fe3aa" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 37563 } } peers { permanent_uuid: "b1dfc55b1eb94835b2dc704f702bd04d" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 33317 } } peers { permanent_uuid: "8f196d39207e4dc4aa3af37234966c83" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 38487 } }
I20260929 10:33:26.148617 20267 leader_election.cc:288] T b158daabef7342ff9769bc5ed2f20f44 P 8f196d39207e4dc4aa3af37234966c83 [CANDIDATE]: Term 1 election: Requested vote from peers bf04219f4e774182abcab2f1546fe3aa (127.15.82.65:37563), b1dfc55b1eb94835b2dc704f702bd04d (127.15.82.66:33317)
I20260929 10:33:26.149039 19932 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b158daabef7342ff9769bc5ed2f20f44" candidate_uuid: "8f196d39207e4dc4aa3af37234966c83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf04219f4e774182abcab2f1546fe3aa"
I20260929 10:33:26.149075 20065 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b158daabef7342ff9769bc5ed2f20f44" candidate_uuid: "8f196d39207e4dc4aa3af37234966c83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1dfc55b1eb94835b2dc704f702bd04d"
I20260929 10:33:26.149195 20065 raft_consensus.cc:3037] T b158daabef7342ff9769bc5ed2f20f44 P b1dfc55b1eb94835b2dc704f702bd04d [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:33:26.149201 19932 raft_consensus.cc:3037] T b158daabef7342ff9769bc5ed2f20f44 P bf04219f4e774182abcab2f1546fe3aa [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:33:26.150451 20065 raft_consensus.cc:2454] T b158daabef7342ff9769bc5ed2f20f44 P b1dfc55b1eb94835b2dc704f702bd04d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f196d39207e4dc4aa3af37234966c83 in term 1.
I20260929 10:33:26.150761 20133 leader_election.cc:302] T b158daabef7342ff9769bc5ed2f20f44 P 8f196d39207e4dc4aa3af37234966c83 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f196d39207e4dc4aa3af37234966c83, b1dfc55b1eb94835b2dc704f702bd04d; no voters:
I20260929 10:33:26.150780 19932 raft_consensus.cc:2454] T b158daabef7342ff9769bc5ed2f20f44 P bf04219f4e774182abcab2f1546fe3aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f196d39207e4dc4aa3af37234966c83 in term 1.
I20260929 10:33:26.150981 20267 raft_consensus.cc:2781] T b158daabef7342ff9769bc5ed2f20f44 P 8f196d39207e4dc4aa3af37234966c83 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:33:26.151203 20267 raft_consensus.cc:686] T b158daabef7342ff9769bc5ed2f20f44 P 8f196d39207e4dc4aa3af37234966c83 [term 1 LEADER]: Becoming Leader. State: Replica: 8f196d39207e4dc4aa3af37234966c83, State: Running, Role: LEADER
I20260929 10:33:26.151610 20267 consensus_queue.cc:237] T b158daabef7342ff9769bc5ed2f20f44 P 8f196d39207e4dc4aa3af37234966c83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf04219f4e774182abcab2f1546fe3aa" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 37563 } } peers { permanent_uuid: "b1dfc55b1eb94835b2dc704f702bd04d" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 33317 } } peers { permanent_uuid: "8f196d39207e4dc4aa3af37234966c83" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 38487 } }
I20260929 10:33:26.155843 19788 catalog_manager.cc:5495] T b158daabef7342ff9769bc5ed2f20f44 P 8f196d39207e4dc4aa3af37234966c83 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f196d39207e4dc4aa3af37234966c83 (127.15.82.67). New cstate: current_term: 1 leader_uuid: "8f196d39207e4dc4aa3af37234966c83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf04219f4e774182abcab2f1546fe3aa" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 37563 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1dfc55b1eb94835b2dc704f702bd04d" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 33317 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f196d39207e4dc4aa3af37234966c83" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 38487 } health_report { overall_health: HEALTHY } } }
I20260929 10:33:26.185384 20260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:26.182142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48050 (local address 127.15.82.67:38487)
0929 10:33:26.182261 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:26.182266 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:26.182280 (+ 14us) server_negotiation.cc:407] Connection header received
0929 10:33:26.182354 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:26.182357 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:26.182458 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:26.182555 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:33:26.182768 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:26.183253 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:26.184053 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:26.184269 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:26.184354 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:26.184626 (+ 272us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:33:26.185041 (+ 415us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:33:26.185131 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:26.185204 (+ 73us) server_negotiation.cc:299] Negotiation successful
0929 10:33:26.185252 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260929 10:33:26.191093 19932 raft_consensus.cc:1261] T b158daabef7342ff9769bc5ed2f20f44 P bf04219f4e774182abcab2f1546fe3aa [term 1 FOLLOWER]: Refusing update from remote peer 8f196d39207e4dc4aa3af37234966c83: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:33:26.191107 20065 raft_consensus.cc:1261] T b158daabef7342ff9769bc5ed2f20f44 P b1dfc55b1eb94835b2dc704f702bd04d [term 1 FOLLOWER]: Refusing update from remote peer 8f196d39207e4dc4aa3af37234966c83: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:33:26.191809 20267 consensus_queue.cc:1035] T b158daabef7342ff9769bc5ed2f20f44 P 8f196d39207e4dc4aa3af37234966c83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1dfc55b1eb94835b2dc704f702bd04d" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 33317 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:33:26.192034 20271 consensus_queue.cc:1035] T b158daabef7342ff9769bc5ed2f20f44 P 8f196d39207e4dc4aa3af37234966c83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf04219f4e774182abcab2f1546fe3aa" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 37563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260929 10:33:26.199097 20245 tablet.cc:2369] T b158daabef7342ff9769bc5ed2f20f44 P 8f196d39207e4dc4aa3af37234966c83: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 10:33:26.202360 20274 mvcc.cc:205] Tried to move back new op lower bound from 7334617113351643136 to 7334617113199980544. Current Snapshot: MvccSnapshot[applied={T|T < 7334617113351643136 or (T in {7334617113351643136})}]
W20260929 10:33:26.204955 20112 tablet.cc:2369] T b158daabef7342ff9769bc5ed2f20f44 P b1dfc55b1eb94835b2dc704f702bd04d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 10:33:26.206890 20275 mvcc.cc:205] Tried to move back new op lower bound from 7334617113351643136 to 7334617113199980544. Current Snapshot: MvccSnapshot[applied={T|T < 7334617113351643136}]
I20260929 10:33:26.208689 20276 mvcc.cc:205] Tried to move back new op lower bound from 7334617113351643136 to 7334617113199980544. Current Snapshot: MvccSnapshot[applied={T|T < 7334617113351643136 or (T in {7334617113351643136})}]
I20260929 10:33:26.213572 19787 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47380:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:33:26.213740 19787 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47380:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:33:26.216466 19787 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a1faa1af1fae4cd9860fb4b0cf366694: Sending DeleteTablet for 3 replicas of tablet b158daabef7342ff9769bc5ed2f20f44
I20260929 10:33:26.217576 19912 tablet_service.cc:1514] Processing DeleteTablet for tablet b158daabef7342ff9769bc5ed2f20f44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:33:26 UTC) from {username='kudu', principal='kudu/127.15.82.126@KRBTEST.COM'} at 127.0.0.1:37388
I20260929 10:33:26.217620 20178 tablet_service.cc:1514] Processing DeleteTablet for tablet b158daabef7342ff9769bc5ed2f20f44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:33:26 UTC) from {username='kudu', principal='kudu/127.15.82.126@KRBTEST.COM'} at 127.0.0.1:48042
I20260929 10:33:26.218075 20283 tablet_replica.cc:330] T b158daabef7342ff9769bc5ed2f20f44 P bf04219f4e774182abcab2f1546fe3aa: stopping tablet replica
I20260929 10:33:26.218076 20043 tablet_service.cc:1514] Processing DeleteTablet for tablet b158daabef7342ff9769bc5ed2f20f44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:33:26 UTC) from {username='kudu', principal='kudu/127.15.82.126@KRBTEST.COM'} at 127.0.0.1:45346
I20260929 10:33:26.218446 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 19850
I20260929 10:33:26.218474 20283 raft_consensus.cc:2229] T b158daabef7342ff9769bc5ed2f20f44 P bf04219f4e774182abcab2f1546fe3aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:33:26.218854 20283 raft_consensus.cc:2258] T b158daabef7342ff9769bc5ed2f20f44 P bf04219f4e774182abcab2f1546fe3aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:33:26.219851 20284 tablet_replica.cc:330] T b158daabef7342ff9769bc5ed2f20f44 P 8f196d39207e4dc4aa3af37234966c83: stopping tablet replica
I20260929 10:33:26.220191 20284 raft_consensus.cc:2229] T b158daabef7342ff9769bc5ed2f20f44 P 8f196d39207e4dc4aa3af37234966c83 [term 1 LEADER]: Raft consensus shutting down.
I20260929 10:33:26.220552 20284 raft_consensus.cc:2258] T b158daabef7342ff9769bc5ed2f20f44 P 8f196d39207e4dc4aa3af37234966c83 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:33:26.222604 20284 ts_tablet_manager.cc:1905] T b158daabef7342ff9769bc5ed2f20f44 P 8f196d39207e4dc4aa3af37234966c83: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:33:26.225437 20284 ts_tablet_manager.cc:1918] T b158daabef7342ff9769bc5ed2f20f44 P 8f196d39207e4dc4aa3af37234966c83: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:33:26.225533 20284 log.cc:1196] T b158daabef7342ff9769bc5ed2f20f44 P 8f196d39207e4dc4aa3af37234966c83: Deleting WAL directory at /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790677981592896-15689-0/minicluster-data/ts-2/wal/wals/b158daabef7342ff9769bc5ed2f20f44
I20260929 10:33:26.225845 20284 ts_tablet_manager.cc:1939] T b158daabef7342ff9769bc5ed2f20f44 P 8f196d39207e4dc4aa3af37234966c83: Deleting consensus metadata
I20260929 10:33:26.227424 19774 catalog_manager.cc:4841] TS 8f196d39207e4dc4aa3af37234966c83 (127.15.82.67:38487): tablet b158daabef7342ff9769bc5ed2f20f44 (table test-table [id=83c79fb51dcc4320a0e97c2ffe0958ac]) successfully deleted
W20260929 10:33:26.229799 19774 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.82.65:37563 (error 108)
W20260929 10:33:26.237066 20257 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:33:26.230622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.65:37563 (local address 127.0.0.1:37404)
0929 10:33:26.230742 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:26.230758 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:26.230848 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:26.236938 (+ 6090us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.65:37563: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":35}
I20260929 10:33:26.237265 20285 tablet_replica.cc:330] T b158daabef7342ff9769bc5ed2f20f44 P b1dfc55b1eb94835b2dc704f702bd04d: stopping tablet replica
W20260929 10:33:26.237347 19774 catalog_manager.cc:4584] TS bf04219f4e774182abcab2f1546fe3aa (127.15.82.65:37563): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b158daabef7342ff9769bc5ed2f20f44: Network error: Client connection negotiation failed: client connection to 127.15.82.65:37563: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260929 10:33:26.237671 20285 raft_consensus.cc:2229] T b158daabef7342ff9769bc5ed2f20f44 P b1dfc55b1eb94835b2dc704f702bd04d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:33:26.238125 20285 raft_consensus.cc:2258] T b158daabef7342ff9769bc5ed2f20f44 P b1dfc55b1eb94835b2dc704f702bd04d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:33:26.238745 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 19983
I20260929 10:33:26.247390 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 20116
I20260929 10:33:26.248343 20257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:26.247550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.66:33317 (local address 127.0.0.1:45358)
0929 10:33:26.248189 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:26.248266 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.66:33317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":549}
I20260929 10:33:26.256347 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 19757
2026-09-29T10:33:26Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2419 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-29T10:33:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:33:26Z Disabled control of system clock
I20260929 10:33:26.287835 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790677981592896-15689-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790677981592896-15689-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790677981592896-15689-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:33835
--webserver_interface=127.15.82.126
--webserver_port=0
--builtin_ntp_servers=127.15.82.84:41213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.82.126:33835
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260929 10:33:26.398389 20290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:33:26.398710 20290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:33:26.398751 20290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:33:26.402886 20290 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:33:26.402946 20290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:33:26.402962 20290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:33:26.402980 20290 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:33:26.403020 20290 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:33:26.407778 20290 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:41213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790677981592896-15689-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790677981592896-15689-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.82.126:33835
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:33835
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.82.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20290
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790677981592896-15689-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:33:26.408880 20290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:33:26.410272 20290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:33:26.418753 20298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:26.418766 20295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:26.418834 20290 server_base.cc:1034] running on GCE node
W20260929 10:33:26.418751 20296 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:26.419727 20290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:33:26.420828 20290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:33:26.422004 20290 hybrid_clock.cc:648] HybridClock initialized: now 1790678006421990 us; error 27 us; skew 500 ppm
I20260929 10:33:26.423972 20290 webserver.cc:460] Webserver started at http://127.15.82.126:39861/ using document root <none> and password file <none>
I20260929 10:33:26.424556 20290 fs_manager.cc:359] Metadata directory not provided
I20260929 10:33:26.424618 20290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:33:26.424852 20290 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:33:26.426996 20290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790677981592896-15689-0/minicluster-data/master-0/data/instance:
uuid: "e2f04abb95c64590a0c9456f550837a4"
format_stamp: "Formatted at 2026-09-29 10:33:26 on dist-test-slave-vg89"
server_key: "e786711d4f654560abe0b80fca99920c"
server_key_iv: "c2100a9a857396db122075e496ff75b9"
server_key_version: "encryptionkey@0"
I20260929 10:33:26.427572 20290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790677981592896-15689-0/minicluster-data/master-0/wal/instance:
uuid: "e2f04abb95c64590a0c9456f550837a4"
format_stamp: "Formatted at 2026-09-29 10:33:26 on dist-test-slave-vg89"
server_key: "e786711d4f654560abe0b80fca99920c"
server_key_iv: "c2100a9a857396db122075e496ff75b9"
server_key_version: "encryptionkey@0"
I20260929 10:33:26.431156 20290 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260929 10:33:26.433463 20304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:26.434794 20290 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 10:33:26.434924 20290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790677981592896-15689-0/minicluster-data/master-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790677981592896-15689-0/minicluster-data/master-0/wal
uuid: "e2f04abb95c64590a0c9456f550837a4"
format_stamp: "Formatted at 2026-09-29 10:33:26 on dist-test-slave-vg89"
server_key: "e786711d4f654560abe0b80fca99920c"
server_key_iv: "c2100a9a857396db122075e496ff75b9"
server_key_version: "encryptionkey@0"
I20260929 10:33:26.435024 20290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790677981592896-15689-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790677981592896-15689-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790677981592896-15689-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:33:26.439539 20290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:33:26.440148 20290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:33:26.440341 20290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:33:26.447191 20290 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.126:33835
I20260929 10:33:26.447196 20356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.126:33835 every 8 connection(s)
I20260929 10:33:26.448134 20290 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
I20260929 10:33:26.451213 20357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:33:26.452958 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 20290
I20260929 10:33:26.453075 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790677981592896-15689-0/minicluster-data/master-0/wal/instance
I20260929 10:33:26.453352 15689 external_mini_cluster.cc:1428] Setting key cdac5b37654f6f4a81ca9225e0b3b826
I20260929 10:33:26.458037 20357 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2f04abb95c64590a0c9456f550837a4: Bootstrap starting.
I20260929 10:33:26.458870 20360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:26.454731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48894 (local address 127.15.82.126:33835)
0929 10:33:26.455315 (+ 584us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:26.455330 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:26.455364 (+ 34us) server_negotiation.cc:407] Connection header received
0929 10:33:26.456008 (+ 644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:26.456025 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:26.456361 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:26.456696 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:26.457505 (+ 809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:26.457510 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:26.457522 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:33:26.457532 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:26.457586 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:33:26.457602 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:26.457751 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:26.457834 (+ 83us) server_negotiation.cc:299] Negotiation successful
0929 10:33:26.458081 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":99,"threads_started":1}
I20260929 10:33:26.460503 20357 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e2f04abb95c64590a0c9456f550837a4: Neither blocks nor log segments found. Creating new log.
I20260929 10:33:26.461326 20357 log.cc:824] T 00000000000000000000000000000000 P e2f04abb95c64590a0c9456f550837a4: Log is configured to *not* fsync() on all Append() calls
I20260929 10:33:26.463120 20357 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2f04abb95c64590a0c9456f550837a4: No bootstrap required, opened a new log
I20260929 10:33:26.466699 20357 raft_consensus.cc:348] T 00000000000000000000000000000000 P e2f04abb95c64590a0c9456f550837a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2f04abb95c64590a0c9456f550837a4" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 33835 } }
I20260929 10:33:26.466938 20357 raft_consensus.cc:374] T 00000000000000000000000000000000 P e2f04abb95c64590a0c9456f550837a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:33:26.467017 20357 raft_consensus.cc:729] T 00000000000000000000000000000000 P e2f04abb95c64590a0c9456f550837a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2f04abb95c64590a0c9456f550837a4, State: Initialized, Role: FOLLOWER
I20260929 10:33:26.467494 20357 consensus_queue.cc:260] T 00000000000000000000000000000000 P e2f04abb95c64590a0c9456f550837a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2f04abb95c64590a0c9456f550837a4" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 33835 } }
I20260929 10:33:26.467644 20357 raft_consensus.cc:388] T 00000000000000000000000000000000 P e2f04abb95c64590a0c9456f550837a4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:33:26.467720 20357 raft_consensus.cc:482] T 00000000000000000000000000000000 P e2f04abb95c64590a0c9456f550837a4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:33:26.467840 20357 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e2f04abb95c64590a0c9456f550837a4 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:33:26.468778 20357 raft_consensus.cc:504] T 00000000000000000000000000000000 P e2f04abb95c64590a0c9456f550837a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2f04abb95c64590a0c9456f550837a4" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 33835 } }
I20260929 10:33:26.469111 20357 leader_election.cc:302] T 00000000000000000000000000000000 P e2f04abb95c64590a0c9456f550837a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e2f04abb95c64590a0c9456f550837a4; no voters:
I20260929 10:33:26.469425 20357 leader_election.cc:288] T 00000000000000000000000000000000 P e2f04abb95c64590a0c9456f550837a4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:33:26.469553 20362 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e2f04abb95c64590a0c9456f550837a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:33:26.469803 20362 raft_consensus.cc:686] T 00000000000000000000000000000000 P e2f04abb95c64590a0c9456f550837a4 [term 1 LEADER]: Becoming Leader. State: Replica: e2f04abb95c64590a0c9456f550837a4, State: Running, Role: LEADER
I20260929 10:33:26.470096 20362 consensus_queue.cc:237] T 00000000000000000000000000000000 P e2f04abb95c64590a0c9456f550837a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2f04abb95c64590a0c9456f550837a4" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 33835 } }
I20260929 10:33:26.470711 20357 sys_catalog.cc:564] T 00000000000000000000000000000000 P e2f04abb95c64590a0c9456f550837a4 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:33:26.471809 20363 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2f04abb95c64590a0c9456f550837a4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e2f04abb95c64590a0c9456f550837a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2f04abb95c64590a0c9456f550837a4" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 33835 } } }
I20260929 10:33:26.471956 20363 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2f04abb95c64590a0c9456f550837a4 [sys.catalog]: This master's current role is: LEADER
I20260929 10:33:26.472374 20371 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:33:26.472970 20364 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2f04abb95c64590a0c9456f550837a4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e2f04abb95c64590a0c9456f550837a4. Latest consensus state: current_term: 1 leader_uuid: "e2f04abb95c64590a0c9456f550837a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2f04abb95c64590a0c9456f550837a4" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 33835 } } }
I20260929 10:33:26.473063 20364 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2f04abb95c64590a0c9456f550837a4 [sys.catalog]: This master's current role is: LEADER
I20260929 10:33:26.475538 20371 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:33:26.480669 20371 catalog_manager.cc:1347] Generated new cluster ID: ff6e175272fb4b619d596318c36d5336
I20260929 10:33:26.480744 20371 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:33:26.493932 20371 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:33:26.494246 20371 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:33:26.503028 20371 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e2f04abb95c64590a0c9456f550837a4: Generated new TSK 0
I20260929 10:33:26.503814 20371 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 10:33:26.516745 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790677981592896-15689-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790677981592896-15689-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790677981592896-15689-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.65:0
--local_ip_for_outbound_sockets=127.15.82.65
--webserver_interface=127.15.82.65
--webserver_port=0
--tserver_master_addrs=127.15.82.126:33835
--builtin_ntp_servers=127.15.82.84:41213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260929 10:33:26.635151 20381 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:33:26.635407 20381 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:33:26.635447 20381 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:33:26.639400 20381 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:33:26.639449 20381 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:33:26.639506 20381 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.65
I20260929 10:33:26.644174 20381 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:41213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790677981592896-15689-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790677981592896-15689-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.82.65
--webserver_port=0
--tserver_master_addrs=127.15.82.126:33835
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20381
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.65
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790677981592896-15689-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:33:26.645170 20381 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:33:26.646498 20381 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:33:26.655838 20387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:26.655857 20386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:26.655841 20389 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:26.656221 20381 server_base.cc:1034] running on GCE node
I20260929 10:33:26.656667 20381 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:33:26.657284 20381 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:33:26.658478 20381 hybrid_clock.cc:648] HybridClock initialized: now 1790678006658435 us; error 45 us; skew 500 ppm
I20260929 10:33:26.660192 20381 webserver.cc:460] Webserver started at http://127.15.82.65:36221/ using document root <none> and password file <none>
I20260929 10:33:26.660725 20381 fs_manager.cc:359] Metadata directory not provided
I20260929 10:33:26.660786 20381 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:33:26.660945 20381 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:33:26.662988 20381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790677981592896-15689-0/minicluster-data/ts-0/data/instance:
uuid: "734a32266f49438ba3d128f7528848eb"
format_stamp: "Formatted at 2026-09-29 10:33:26 on dist-test-slave-vg89"
server_key: "4540964f710b204efeb197df8422e25d"
server_key_iv: "f5ba9a0c958acfeb656ef4634e93c776"
server_key_version: "encryptionkey@0"
I20260929 10:33:26.663484 20381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790677981592896-15689-0/minicluster-data/ts-0/wal/instance:
uuid: "734a32266f49438ba3d128f7528848eb"
format_stamp: "Formatted at 2026-09-29 10:33:26 on dist-test-slave-vg89"
server_key: "4540964f710b204efeb197df8422e25d"
server_key_iv: "f5ba9a0c958acfeb656ef4634e93c776"
server_key_version: "encryptionkey@0"
I20260929 10:33:26.667052 20381 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260929 10:33:26.669548 20395 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:26.670833 20381 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260929 10:33:26.670958 20381 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790677981592896-15689-0/minicluster-data/ts-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790677981592896-15689-0/minicluster-data/ts-0/wal
uuid: "734a32266f49438ba3d128f7528848eb"
format_stamp: "Formatted at 2026-09-29 10:33:26 on dist-test-slave-vg89"
server_key: "4540964f710b204efeb197df8422e25d"
server_key_iv: "f5ba9a0c958acfeb656ef4634e93c776"
server_key_version: "encryptionkey@0"
I20260929 10:33:26.671069 20381 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790677981592896-15689-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790677981592896-15689-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790677981592896-15689-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:33:26.675704 20381 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:33:26.676311 20381 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:33:26.676492 20381 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:33:26.677029 20381 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:33:26.677899 20381 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:33:26.677968 20381 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:26.678037 20381 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:33:26.678079 20381 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:26.686754 20381 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.65:42985
I20260929 10:33:26.686887 20508 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.65:42985 every 8 connection(s)
I20260929 10:33:26.687938 20381 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
I20260929 10:33:26.693797 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 20381
I20260929 10:33:26.693933 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790677981592896-15689-0/minicluster-data/ts-0/wal/instance
I20260929 10:33:26.693956 20360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:26.689779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.65:41843 (local address 127.15.82.126:33835)
0929 10:33:26.689916 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:26.689919 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:26.690958 (+ 1039us) server_negotiation.cc:407] Connection header received
0929 10:33:26.692278 (+ 1320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:26.692282 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:26.692330 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:26.692415 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:26.693467 (+ 1052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:26.693471 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:26.693473 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:33:26.693486 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:26.693510 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:33:26.693525 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:26.693623 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:26.693777 (+ 154us) server_negotiation.cc:299] Negotiation successful
0929 10:33:26.693824 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260929 10:33:26.694233 15689 external_mini_cluster.cc:1428] Setting key 6f6abc655b210a64d49bbdf5ae08c877
I20260929 10:33:26.695032 20511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:26.690101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:33835 (local address 127.15.82.65:41843)
0929 10:33:26.690808 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:26.690847 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:26.692051 (+ 1204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:26.692590 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:26.692597 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:26.692683 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:26.693193 (+ 510us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 10:33:26.693194 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:26.693234 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 10:33:26.693319 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:26.693607 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:26.693615 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:26.693620 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 10:33:26.693802 (+ 182us) client_negotiation.cc:240] Negotiation successful
0929 10:33:26.694040 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":111,"threads_started":1}
I20260929 10:33:26.696221 20509 heartbeater.cc:347] Connected to a master server at 127.15.82.126:33835
I20260929 10:33:26.696475 20509 heartbeater.cc:464] Registering TS with master...
I20260929 10:33:26.696481 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790677981592896-15689-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790677981592896-15689-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790677981592896-15689-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.66:0
--local_ip_for_outbound_sockets=127.15.82.66
--webserver_interface=127.15.82.66
--webserver_port=0
--tserver_master_addrs=127.15.82.126:33835
--builtin_ntp_servers=127.15.82.84:41213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260929 10:33:26.696928 20509 heartbeater.cc:511] Master 127.15.82.126:33835 requested a full tablet report, sending...
I20260929 10:33:26.698624 20321 ts_manager.cc:194] Registered new tserver with Master: 734a32266f49438ba3d128f7528848eb (127.15.82.65:42985)
W20260929 10:33:26.805069 20512 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:33:26.805305 20512 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:33:26.805342 20512 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:33:26.809213 20512 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:33:26.809260 20512 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:33:26.809314 20512 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.66
I20260929 10:33:26.813827 20512 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:41213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790677981592896-15689-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790677981592896-15689-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.82.66
--webserver_port=0
--tserver_master_addrs=127.15.82.126:33835
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20512
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.66
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790677981592896-15689-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:33:26.814836 20512 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:33:26.816139 20512 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:33:26.825744 20518 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:26.825850 20512 server_base.cc:1034] running on GCE node
W20260929 10:33:26.825744 20517 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:26.825783 20520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:26.826519 20512 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:33:26.827128 20512 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:33:26.828286 20512 hybrid_clock.cc:648] HybridClock initialized: now 1790678006828271 us; error 28 us; skew 500 ppm
I20260929 10:33:26.830000 20512 webserver.cc:460] Webserver started at http://127.15.82.66:40653/ using document root <none> and password file <none>
I20260929 10:33:26.830683 20512 fs_manager.cc:359] Metadata directory not provided
I20260929 10:33:26.830746 20512 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:33:26.830982 20512 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:33:26.832991 20512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790677981592896-15689-0/minicluster-data/ts-1/data/instance:
uuid: "f56c1806329847c2aa32e3ef2fdcbaaf"
format_stamp: "Formatted at 2026-09-29 10:33:26 on dist-test-slave-vg89"
server_key: "257949325e55ed37032a8a9dace5a270"
server_key_iv: "da746f87fd63dd4dd013ee8f93d3ca17"
server_key_version: "encryptionkey@0"
I20260929 10:33:26.833554 20512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790677981592896-15689-0/minicluster-data/ts-1/wal/instance:
uuid: "f56c1806329847c2aa32e3ef2fdcbaaf"
format_stamp: "Formatted at 2026-09-29 10:33:26 on dist-test-slave-vg89"
server_key: "257949325e55ed37032a8a9dace5a270"
server_key_iv: "da746f87fd63dd4dd013ee8f93d3ca17"
server_key_version: "encryptionkey@0"
I20260929 10:33:26.837123 20512 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260929 10:33:26.839780 20526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:26.840950 20512 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260929 10:33:26.841065 20512 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790677981592896-15689-0/minicluster-data/ts-1/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790677981592896-15689-0/minicluster-data/ts-1/wal
uuid: "f56c1806329847c2aa32e3ef2fdcbaaf"
format_stamp: "Formatted at 2026-09-29 10:33:26 on dist-test-slave-vg89"
server_key: "257949325e55ed37032a8a9dace5a270"
server_key_iv: "da746f87fd63dd4dd013ee8f93d3ca17"
server_key_version: "encryptionkey@0"
I20260929 10:33:26.841166 20512 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790677981592896-15689-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790677981592896-15689-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790677981592896-15689-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:33:26.845588 20512 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:33:26.846150 20512 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:33:26.846339 20512 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:33:26.847076 20512 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:33:26.848086 20512 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:33:26.848155 20512 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:26.848215 20512 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:33:26.848253 20512 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:26.856937 20512 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.66:35589
I20260929 10:33:26.856956 20639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.66:35589 every 8 connection(s)
I20260929 10:33:26.857779 20512 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
I20260929 10:33:26.863348 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 20512
I20260929 10:33:26.863476 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790677981592896-15689-0/minicluster-data/ts-1/wal/instance
I20260929 10:33:26.863813 15689 external_mini_cluster.cc:1428] Setting key 0f536318747fc71d2900a0b786cf885a
I20260929 10:33:26.864118 20360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:26.859591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:55131 (local address 127.15.82.126:33835)
0929 10:33:26.859733 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:26.859737 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:26.860759 (+ 1022us) server_negotiation.cc:407] Connection header received
0929 10:33:26.862068 (+ 1309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:26.862073 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:26.862119 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:26.862224 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:26.863668 (+ 1444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:26.863672 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:26.863675 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:33:26.863694 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:26.863714 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:33:26.863723 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:26.863805 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:26.863972 (+ 167us) server_negotiation.cc:299] Negotiation successful
0929 10:33:26.864011 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260929 10:33:26.865006 20642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:26.859981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:33835 (local address 127.15.82.66:55131)
0929 10:33:26.860616 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:26.860654 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:26.861862 (+ 1208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:26.862383 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:26.862390 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:26.862547 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:26.863274 (+ 727us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 10:33:26.863276 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:26.863333 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 10:33:26.863423 (+ 90us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:26.863817 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:26.863828 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:26.863836 (+ 8us) client_negotiation.cc:769] Sending connection context
0929 10:33:26.864014 (+ 178us) client_negotiation.cc:240] Negotiation successful
0929 10:33:26.864244 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":97,"threads_started":1}
I20260929 10:33:26.865844 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790677981592896-15689-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790677981592896-15689-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790677981592896-15689-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.67:0
--local_ip_for_outbound_sockets=127.15.82.67
--webserver_interface=127.15.82.67
--webserver_port=0
--tserver_master_addrs=127.15.82.126:33835
--builtin_ntp_servers=127.15.82.84:41213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260929 10:33:26.866094 20640 heartbeater.cc:347] Connected to a master server at 127.15.82.126:33835
I20260929 10:33:26.866308 20640 heartbeater.cc:464] Registering TS with master...
I20260929 10:33:26.866792 20640 heartbeater.cc:511] Master 127.15.82.126:33835 requested a full tablet report, sending...
I20260929 10:33:26.867827 20321 ts_manager.cc:194] Registered new tserver with Master: f56c1806329847c2aa32e3ef2fdcbaaf (127.15.82.66:35589)
W20260929 10:33:26.976423 20643 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:33:26.976676 20643 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:33:26.976715 20643 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:33:26.980741 20643 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:33:26.980798 20643 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:33:26.980852 20643 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.67
I20260929 10:33:26.985440 20643 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:41213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790677981592896-15689-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790677981592896-15689-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.82.67
--webserver_port=0
--tserver_master_addrs=127.15.82.126:33835
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20643
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.67
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790677981592896-15689-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:33:26.986442 20643 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:33:26.987784 20643 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:33:26.997113 20651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:26.997097 20648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:26.997179 20649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:26.997278 20643 server_base.cc:1034] running on GCE node
I20260929 10:33:26.997706 20643 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:33:26.998301 20643 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:33:26.999483 20643 hybrid_clock.cc:648] HybridClock initialized: now 1790678006999456 us; error 48 us; skew 500 ppm
I20260929 10:33:27.001166 20643 webserver.cc:460] Webserver started at http://127.15.82.67:40849/ using document root <none> and password file <none>
I20260929 10:33:27.001729 20643 fs_manager.cc:359] Metadata directory not provided
I20260929 10:33:27.001823 20643 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:33:27.002046 20643 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:33:27.004189 20643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790677981592896-15689-0/minicluster-data/ts-2/data/instance:
uuid: "6d38cd968e41446f908ccaf258e1045b"
format_stamp: "Formatted at 2026-09-29 10:33:26 on dist-test-slave-vg89"
server_key: "a57aa141064fbd67be54689d629de3b1"
server_key_iv: "8d9f54e8e16f70d206d036faf0a84d0d"
server_key_version: "encryptionkey@0"
I20260929 10:33:27.004765 20643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790677981592896-15689-0/minicluster-data/ts-2/wal/instance:
uuid: "6d38cd968e41446f908ccaf258e1045b"
format_stamp: "Formatted at 2026-09-29 10:33:26 on dist-test-slave-vg89"
server_key: "a57aa141064fbd67be54689d629de3b1"
server_key_iv: "8d9f54e8e16f70d206d036faf0a84d0d"
server_key_version: "encryptionkey@0"
I20260929 10:33:27.008479 20643 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260929 10:33:27.011024 20657 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:27.012195 20643 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 10:33:27.012347 20643 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790677981592896-15689-0/minicluster-data/ts-2/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790677981592896-15689-0/minicluster-data/ts-2/wal
uuid: "6d38cd968e41446f908ccaf258e1045b"
format_stamp: "Formatted at 2026-09-29 10:33:26 on dist-test-slave-vg89"
server_key: "a57aa141064fbd67be54689d629de3b1"
server_key_iv: "8d9f54e8e16f70d206d036faf0a84d0d"
server_key_version: "encryptionkey@0"
I20260929 10:33:27.012465 20643 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790677981592896-15689-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790677981592896-15689-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790677981592896-15689-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:33:27.017138 20643 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:33:27.017727 20643 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:33:27.017907 20643 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:33:27.018486 20643 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:33:27.019389 20643 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:33:27.019459 20643 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:27.019526 20643 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:33:27.019577 20643 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:27.028261 20643 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.67:43181
I20260929 10:33:27.028286 20770 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.67:43181 every 8 connection(s)
I20260929 10:33:27.029121 20643 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
I20260929 10:33:27.032402 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 20643
I20260929 10:33:27.032534 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790677981592896-15689-0/minicluster-data/ts-2/wal/instance
I20260929 10:33:27.032812 15689 external_mini_cluster.cc:1428] Setting key 8f508b6b2c65974d947e42b748b7c99b
I20260929 10:33:27.035379 20360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:27.030946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.67:35945 (local address 127.15.82.126:33835)
0929 10:33:27.031091 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:27.031095 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:27.032104 (+ 1009us) server_negotiation.cc:407] Connection header received
0929 10:33:27.033652 (+ 1548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:27.033655 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:27.033697 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:27.033803 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:27.034913 (+ 1110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:27.034916 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:27.034919 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:33:27.034930 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:27.034953 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:33:27.034969 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:27.035056 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:27.035206 (+ 150us) server_negotiation.cc:299] Negotiation successful
0929 10:33:27.035254 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260929 10:33:27.036166 20773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:27.031329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:33835 (local address 127.15.82.67:35945)
0929 10:33:27.031955 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:27.031994 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:27.033452 (+ 1458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:27.033961 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:27.033967 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:27.034053 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:27.034656 (+ 603us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 10:33:27.034657 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:27.034696 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 10:33:27.034770 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:27.035056 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:27.035067 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:27.035075 (+ 8us) client_negotiation.cc:769] Sending connection context
0929 10:33:27.035248 (+ 173us) client_negotiation.cc:240] Negotiation successful
0929 10:33:27.035477 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":102,"threads_started":1}
I20260929 10:33:27.037154 20771 heartbeater.cc:347] Connected to a master server at 127.15.82.126:33835
I20260929 10:33:27.037345 20771 heartbeater.cc:464] Registering TS with master...
I20260929 10:33:27.037671 20771 heartbeater.cc:511] Master 127.15.82.126:33835 requested a full tablet report, sending...
I20260929 10:33:27.038601 20321 ts_manager.cc:194] Registered new tserver with Master: 6d38cd968e41446f908ccaf258e1045b (127.15.82.67:43181)
I20260929 10:33:27.046535 15689 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 10:33:27.049854 20360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:27.048403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48902 (local address 127.15.82.126:33835)
0929 10:33:27.048531 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:27.048535 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:27.048711 (+ 176us) server_negotiation.cc:407] Connection header received
0929 10:33:27.048811 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:27.048813 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:27.048843 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:27.048908 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:27.049588 (+ 680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:27.049591 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:27.049593 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:33:27.049600 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:27.049615 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:33:27.049622 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:27.049682 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:27.049727 (+ 45us) server_negotiation.cc:299] Negotiation successful
0929 10:33:27.049758 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260929 10:33:27.052600 20321 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48902:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 10:33:27.055476 20321 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:33:27.066689 20784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:27.063435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58142 (local address 127.15.82.66:35589)
0929 10:33:27.063779 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:27.063790 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:27.064177 (+ 387us) server_negotiation.cc:407] Connection header received
0929 10:33:27.064246 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:27.064272 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:27.064468 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:27.064610 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:27.065778 (+ 1168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:27.065785 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:27.065801 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:33:27.065813 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:27.065861 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:33:27.065886 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:27.066002 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:27.066490 (+ 488us) server_negotiation.cc:299] Negotiation successful
0929 10:33:27.066568 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":232,"thread_start_us":126,"threads_started":1}
I20260929 10:33:27.066953 20783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:27.063391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.67:43181 (local address 127.0.0.1:38048)
0929 10:33:27.064649 (+ 1258us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:27.064661 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:27.064741 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:27.065231 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:27.065234 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:27.065250 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:27.065657 (+ 407us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 10:33:27.065661 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:27.065673 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 10:33:27.065700 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:27.066048 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:27.066057 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:27.066060 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 10:33:27.066174 (+ 114us) client_negotiation.cc:240] Negotiation successful
0929 10:33:27.066237 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1197,"thread_start_us":100,"threads_started":1}
I20260929 10:33:27.067284 20782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:27.063302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.66:35589 (local address 127.0.0.1:58142)
0929 10:33:27.063922 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:27.063969 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:27.064074 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:27.064879 (+ 805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:27.064884 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:27.064963 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:27.065506 (+ 543us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 10:33:27.065507 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:27.065523 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 10:33:27.065628 (+ 105us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:27.066043 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:27.066053 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:27.066061 (+ 8us) client_negotiation.cc:769] Sending connection context
0929 10:33:27.066170 (+ 109us) client_negotiation.cc:240] Negotiation successful
0929 10:33:27.066231 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":69,"threads_started":1}
I20260929 10:33:27.067337 20786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:27.064066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38048 (local address 127.15.82.67:43181)
0929 10:33:27.064507 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:27.064527 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:27.064764 (+ 237us) server_negotiation.cc:407] Connection header received
0929 10:33:27.064833 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:27.064839 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:27.065009 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:27.065132 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:27.065810 (+ 678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:27.065815 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:27.065825 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:33:27.065836 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:27.065879 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:33:27.065911 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:27.066004 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:27.066320 (+ 316us) server_negotiation.cc:299] Negotiation successful
0929 10:33:27.066420 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":157,"threads_started":1}
I20260929 10:33:27.067394 20785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:27.063999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.65:42985 (local address 127.0.0.1:52824)
0929 10:33:27.064776 (+ 777us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:27.064790 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:27.064860 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:27.065975 (+ 1115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:27.065978 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:27.065991 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:27.066414 (+ 423us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 10:33:27.066416 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:27.066426 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 10:33:27.066453 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:27.067099 (+ 646us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:27.067102 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:27.067105 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 10:33:27.067183 (+ 78us) client_negotiation.cc:240] Negotiation successful
0929 10:33:27.067216 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":689,"thread_start_us":83,"threads_started":1}
I20260929 10:33:27.067807 20787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:27.064397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52824 (local address 127.15.82.65:42985)
0929 10:33:27.065098 (+ 701us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:27.065109 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:27.065130 (+ 21us) server_negotiation.cc:407] Connection header received
0929 10:33:27.065670 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:27.065677 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:27.065839 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:27.065977 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:27.066867 (+ 890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:27.066872 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:27.066881 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:33:27.066892 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:27.066928 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:33:27.066951 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:27.067056 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:27.067215 (+ 159us) server_negotiation.cc:299] Negotiation successful
0929 10:33:27.067282 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588,"thread_start_us":121,"threads_started":1}
I20260929 10:33:27.070142 20705 tablet_service.cc:1467] Processing CreateTablet for tablet 83bed19889b240828cb43f23578df343 (DEFAULT_TABLE table=test-table [id=5fce7a9d4a484a489a8482d76b8de6e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:33:27.070143 20574 tablet_service.cc:1467] Processing CreateTablet for tablet 83bed19889b240828cb43f23578df343 (DEFAULT_TABLE table=test-table [id=5fce7a9d4a484a489a8482d76b8de6e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:33:27.070317 20443 tablet_service.cc:1467] Processing CreateTablet for tablet 83bed19889b240828cb43f23578df343 (DEFAULT_TABLE table=test-table [id=5fce7a9d4a484a489a8482d76b8de6e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:33:27.071245 20705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83bed19889b240828cb43f23578df343. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:33:27.071249 20574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83bed19889b240828cb43f23578df343. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:33:27.071259 20443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83bed19889b240828cb43f23578df343. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:33:27.077471 20788 tablet_bootstrap.cc:492] T 83bed19889b240828cb43f23578df343 P 6d38cd968e41446f908ccaf258e1045b: Bootstrap starting.
I20260929 10:33:27.078801 20790 tablet_bootstrap.cc:492] T 83bed19889b240828cb43f23578df343 P f56c1806329847c2aa32e3ef2fdcbaaf: Bootstrap starting.
I20260929 10:33:27.080188 20788 tablet_bootstrap.cc:654] T 83bed19889b240828cb43f23578df343 P 6d38cd968e41446f908ccaf258e1045b: Neither blocks nor log segments found. Creating new log.
I20260929 10:33:27.080534 20790 tablet_bootstrap.cc:654] T 83bed19889b240828cb43f23578df343 P f56c1806329847c2aa32e3ef2fdcbaaf: Neither blocks nor log segments found. Creating new log.
I20260929 10:33:27.080986 20788 log.cc:824] T 83bed19889b240828cb43f23578df343 P 6d38cd968e41446f908ccaf258e1045b: Log is configured to *not* fsync() on all Append() calls
I20260929 10:33:27.081282 20790 log.cc:824] T 83bed19889b240828cb43f23578df343 P f56c1806329847c2aa32e3ef2fdcbaaf: Log is configured to *not* fsync() on all Append() calls
I20260929 10:33:27.081682 20789 tablet_bootstrap.cc:492] T 83bed19889b240828cb43f23578df343 P 734a32266f49438ba3d128f7528848eb: Bootstrap starting.
I20260929 10:33:27.082876 20788 tablet_bootstrap.cc:492] T 83bed19889b240828cb43f23578df343 P 6d38cd968e41446f908ccaf258e1045b: No bootstrap required, opened a new log
I20260929 10:33:27.083084 20788 ts_tablet_manager.cc:1397] T 83bed19889b240828cb43f23578df343 P 6d38cd968e41446f908ccaf258e1045b: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260929 10:33:27.083189 20790 tablet_bootstrap.cc:492] T 83bed19889b240828cb43f23578df343 P f56c1806329847c2aa32e3ef2fdcbaaf: No bootstrap required, opened a new log
I20260929 10:33:27.083369 20790 ts_tablet_manager.cc:1397] T 83bed19889b240828cb43f23578df343 P f56c1806329847c2aa32e3ef2fdcbaaf: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260929 10:33:27.083871 20789 tablet_bootstrap.cc:654] T 83bed19889b240828cb43f23578df343 P 734a32266f49438ba3d128f7528848eb: Neither blocks nor log segments found. Creating new log.
I20260929 10:33:27.084669 20789 log.cc:824] T 83bed19889b240828cb43f23578df343 P 734a32266f49438ba3d128f7528848eb: Log is configured to *not* fsync() on all Append() calls
I20260929 10:33:27.086726 20789 tablet_bootstrap.cc:492] T 83bed19889b240828cb43f23578df343 P 734a32266f49438ba3d128f7528848eb: No bootstrap required, opened a new log
I20260929 10:33:27.086920 20789 ts_tablet_manager.cc:1397] T 83bed19889b240828cb43f23578df343 P 734a32266f49438ba3d128f7528848eb: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260929 10:33:27.087275 20788 raft_consensus.cc:348] T 83bed19889b240828cb43f23578df343 P 6d38cd968e41446f908ccaf258e1045b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f56c1806329847c2aa32e3ef2fdcbaaf" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 35589 } } peers { permanent_uuid: "6d38cd968e41446f908ccaf258e1045b" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 43181 } } peers { permanent_uuid: "734a32266f49438ba3d128f7528848eb" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 42985 } }
I20260929 10:33:27.087563 20788 raft_consensus.cc:374] T 83bed19889b240828cb43f23578df343 P 6d38cd968e41446f908ccaf258e1045b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:33:27.087617 20788 raft_consensus.cc:729] T 83bed19889b240828cb43f23578df343 P 6d38cd968e41446f908ccaf258e1045b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d38cd968e41446f908ccaf258e1045b, State: Initialized, Role: FOLLOWER
I20260929 10:33:27.088035 20788 consensus_queue.cc:260] T 83bed19889b240828cb43f23578df343 P 6d38cd968e41446f908ccaf258e1045b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f56c1806329847c2aa32e3ef2fdcbaaf" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 35589 } } peers { permanent_uuid: "6d38cd968e41446f908ccaf258e1045b" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 43181 } } peers { permanent_uuid: "734a32266f49438ba3d128f7528848eb" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 42985 } }
I20260929 10:33:27.088011 20790 raft_consensus.cc:348] T 83bed19889b240828cb43f23578df343 P f56c1806329847c2aa32e3ef2fdcbaaf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f56c1806329847c2aa32e3ef2fdcbaaf" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 35589 } } peers { permanent_uuid: "6d38cd968e41446f908ccaf258e1045b" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 43181 } } peers { permanent_uuid: "734a32266f49438ba3d128f7528848eb" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 42985 } }
I20260929 10:33:27.088261 20790 raft_consensus.cc:374] T 83bed19889b240828cb43f23578df343 P f56c1806329847c2aa32e3ef2fdcbaaf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:33:27.088311 20790 raft_consensus.cc:729] T 83bed19889b240828cb43f23578df343 P f56c1806329847c2aa32e3ef2fdcbaaf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f56c1806329847c2aa32e3ef2fdcbaaf, State: Initialized, Role: FOLLOWER
I20260929 10:33:27.088716 20790 consensus_queue.cc:260] T 83bed19889b240828cb43f23578df343 P f56c1806329847c2aa32e3ef2fdcbaaf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f56c1806329847c2aa32e3ef2fdcbaaf" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 35589 } } peers { permanent_uuid: "6d38cd968e41446f908ccaf258e1045b" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 43181 } } peers { permanent_uuid: "734a32266f49438ba3d128f7528848eb" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 42985 } }
I20260929 10:33:27.089032 20788 ts_tablet_manager.cc:1428] T 83bed19889b240828cb43f23578df343 P 6d38cd968e41446f908ccaf258e1045b: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260929 10:33:27.089392 20640 heartbeater.cc:503] Master 127.15.82.126:33835 was elected leader, sending a full tablet report...
I20260929 10:33:27.089392 20771 heartbeater.cc:503] Master 127.15.82.126:33835 was elected leader, sending a full tablet report...
I20260929 10:33:27.089658 20790 ts_tablet_manager.cc:1428] T 83bed19889b240828cb43f23578df343 P f56c1806329847c2aa32e3ef2fdcbaaf: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260929 10:33:27.091148 20789 raft_consensus.cc:348] T 83bed19889b240828cb43f23578df343 P 734a32266f49438ba3d128f7528848eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f56c1806329847c2aa32e3ef2fdcbaaf" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 35589 } } peers { permanent_uuid: "6d38cd968e41446f908ccaf258e1045b" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 43181 } } peers { permanent_uuid: "734a32266f49438ba3d128f7528848eb" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 42985 } }
I20260929 10:33:27.091405 20789 raft_consensus.cc:374] T 83bed19889b240828cb43f23578df343 P 734a32266f49438ba3d128f7528848eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:33:27.091456 20789 raft_consensus.cc:729] T 83bed19889b240828cb43f23578df343 P 734a32266f49438ba3d128f7528848eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 734a32266f49438ba3d128f7528848eb, State: Initialized, Role: FOLLOWER
I20260929 10:33:27.091874 20789 consensus_queue.cc:260] T 83bed19889b240828cb43f23578df343 P 734a32266f49438ba3d128f7528848eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f56c1806329847c2aa32e3ef2fdcbaaf" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 35589 } } peers { permanent_uuid: "6d38cd968e41446f908ccaf258e1045b" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 43181 } } peers { permanent_uuid: "734a32266f49438ba3d128f7528848eb" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 42985 } }
I20260929 10:33:27.092597 20509 heartbeater.cc:503] Master 127.15.82.126:33835 was elected leader, sending a full tablet report...
I20260929 10:33:27.092842 20789 ts_tablet_manager.cc:1428] T 83bed19889b240828cb43f23578df343 P 734a32266f49438ba3d128f7528848eb: Time spent starting tablet: real 0.006s user 0.002s sys 0.003s
W20260929 10:33:27.108764 20641 tablet.cc:2369] T 83bed19889b240828cb43f23578df343 P f56c1806329847c2aa32e3ef2fdcbaaf: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 10:33:27.189090 20510 tablet.cc:2369] T 83bed19889b240828cb43f23578df343 P 734a32266f49438ba3d128f7528848eb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 10:33:27.194738 20795 raft_consensus.cc:482] T 83bed19889b240828cb43f23578df343 P f56c1806329847c2aa32e3ef2fdcbaaf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:33:27.194918 20795 raft_consensus.cc:504] T 83bed19889b240828cb43f23578df343 P f56c1806329847c2aa32e3ef2fdcbaaf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f56c1806329847c2aa32e3ef2fdcbaaf" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 35589 } } peers { permanent_uuid: "6d38cd968e41446f908ccaf258e1045b" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 43181 } } peers { permanent_uuid: "734a32266f49438ba3d128f7528848eb" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 42985 } }
I20260929 10:33:27.196089 20795 leader_election.cc:288] T 83bed19889b240828cb43f23578df343 P f56c1806329847c2aa32e3ef2fdcbaaf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d38cd968e41446f908ccaf258e1045b (127.15.82.67:43181), 734a32266f49438ba3d128f7528848eb (127.15.82.65:42985)
I20260929 10:33:27.198160 20642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:27.196384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.67:43181 (local address 127.15.82.66:55611)
0929 10:33:27.196542 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:27.196560 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:27.196663 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:27.196998 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:27.197002 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:27.197021 (+ 19us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:27.197520 (+ 499us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 10:33:27.197522 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:27.197533 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 10:33:27.197584 (+ 51us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:27.197883 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:27.197885 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:27.197888 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 10:33:27.197988 (+ 100us) client_negotiation.cc:240] Negotiation successful
0929 10:33:27.198024 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260929 10:33:27.198184 20786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:27.196459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:55611 (local address 127.15.82.67:43181)
0929 10:33:27.196605 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:27.196612 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:27.196629 (+ 17us) server_negotiation.cc:407] Connection header received
0929 10:33:27.196821 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:27.196827 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:27.196874 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:27.196963 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:27.197724 (+ 761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:27.197728 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:27.197733 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:33:27.197754 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:27.197775 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:33:27.197787 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:27.197885 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:27.197984 (+ 99us) server_negotiation.cc:299] Negotiation successful
0929 10:33:27.198030 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260929 10:33:27.198375 20787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:27.196804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:55751 (local address 127.15.82.65:42985)
0929 10:33:27.196938 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:27.196942 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:27.197128 (+ 186us) server_negotiation.cc:407] Connection header received
0929 10:33:27.197233 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:27.197237 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:27.197277 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:27.197363 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:27.197981 (+ 618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:27.197985 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:27.197987 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:33:27.198000 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:27.198019 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:33:27.198028 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:27.198109 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:27.198195 (+ 86us) server_negotiation.cc:299] Negotiation successful
0929 10:33:27.198242 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 10:33:27.198442 20797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:27.196665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.65:42985 (local address 127.15.82.66:55751)
0929 10:33:27.197043 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:27.197053 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:27.197156 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:27.197383 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:27.197385 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:27.197399 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:27.197823 (+ 424us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 10:33:27.197827 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:27.197839 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 10:33:27.197866 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:27.198113 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:27.198116 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:27.198120 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 10:33:27.198198 (+ 78us) client_negotiation.cc:240] Negotiation successful
0929 10:33:27.198234 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":160,"threads_started":1}
I20260929 10:33:27.198803 20725 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83bed19889b240828cb43f23578df343" candidate_uuid: "f56c1806329847c2aa32e3ef2fdcbaaf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d38cd968e41446f908ccaf258e1045b" is_pre_election: true
I20260929 10:33:27.199057 20725 raft_consensus.cc:2454] T 83bed19889b240828cb43f23578df343 P 6d38cd968e41446f908ccaf258e1045b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f56c1806329847c2aa32e3ef2fdcbaaf in term 0.
I20260929 10:33:27.199553 20527 leader_election.cc:302] T 83bed19889b240828cb43f23578df343 P f56c1806329847c2aa32e3ef2fdcbaaf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d38cd968e41446f908ccaf258e1045b, f56c1806329847c2aa32e3ef2fdcbaaf; no voters:
I20260929 10:33:27.199991 20795 raft_consensus.cc:2781] T 83bed19889b240828cb43f23578df343 P f56c1806329847c2aa32e3ef2fdcbaaf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:33:27.199949 20463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83bed19889b240828cb43f23578df343" candidate_uuid: "f56c1806329847c2aa32e3ef2fdcbaaf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "734a32266f49438ba3d128f7528848eb" is_pre_election: true
I20260929 10:33:27.200093 20795 raft_consensus.cc:482] T 83bed19889b240828cb43f23578df343 P f56c1806329847c2aa32e3ef2fdcbaaf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:33:27.200138 20795 raft_consensus.cc:3037] T 83bed19889b240828cb43f23578df343 P f56c1806329847c2aa32e3ef2fdcbaaf [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:33:27.200294 20463 raft_consensus.cc:2454] T 83bed19889b240828cb43f23578df343 P 734a32266f49438ba3d128f7528848eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f56c1806329847c2aa32e3ef2fdcbaaf in term 0.
I20260929 10:33:27.201258 20795 raft_consensus.cc:504] T 83bed19889b240828cb43f23578df343 P f56c1806329847c2aa32e3ef2fdcbaaf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f56c1806329847c2aa32e3ef2fdcbaaf" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 35589 } } peers { permanent_uuid: "6d38cd968e41446f908ccaf258e1045b" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 43181 } } peers { permanent_uuid: "734a32266f49438ba3d128f7528848eb" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 42985 } }
I20260929 10:33:27.201543 20795 leader_election.cc:288] T 83bed19889b240828cb43f23578df343 P f56c1806329847c2aa32e3ef2fdcbaaf [CANDIDATE]: Term 1 election: Requested vote from peers 6d38cd968e41446f908ccaf258e1045b (127.15.82.67:43181), 734a32266f49438ba3d128f7528848eb (127.15.82.65:42985)
I20260929 10:33:27.201951 20725 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83bed19889b240828cb43f23578df343" candidate_uuid: "f56c1806329847c2aa32e3ef2fdcbaaf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d38cd968e41446f908ccaf258e1045b"
I20260929 10:33:27.201979 20463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83bed19889b240828cb43f23578df343" candidate_uuid: "f56c1806329847c2aa32e3ef2fdcbaaf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "734a32266f49438ba3d128f7528848eb"
I20260929 10:33:27.202076 20463 raft_consensus.cc:3037] T 83bed19889b240828cb43f23578df343 P 734a32266f49438ba3d128f7528848eb [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:33:27.202106 20725 raft_consensus.cc:3037] T 83bed19889b240828cb43f23578df343 P 6d38cd968e41446f908ccaf258e1045b [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:33:27.203214 20463 raft_consensus.cc:2454] T 83bed19889b240828cb43f23578df343 P 734a32266f49438ba3d128f7528848eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f56c1806329847c2aa32e3ef2fdcbaaf in term 1.
I20260929 10:33:27.203231 20725 raft_consensus.cc:2454] T 83bed19889b240828cb43f23578df343 P 6d38cd968e41446f908ccaf258e1045b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f56c1806329847c2aa32e3ef2fdcbaaf in term 1.
I20260929 10:33:27.203485 20527 leader_election.cc:302] T 83bed19889b240828cb43f23578df343 P f56c1806329847c2aa32e3ef2fdcbaaf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 734a32266f49438ba3d128f7528848eb, f56c1806329847c2aa32e3ef2fdcbaaf; no voters:
I20260929 10:33:27.203663 20795 raft_consensus.cc:2781] T 83bed19889b240828cb43f23578df343 P f56c1806329847c2aa32e3ef2fdcbaaf [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:33:27.203891 20795 raft_consensus.cc:686] T 83bed19889b240828cb43f23578df343 P f56c1806329847c2aa32e3ef2fdcbaaf [term 1 LEADER]: Becoming Leader. State: Replica: f56c1806329847c2aa32e3ef2fdcbaaf, State: Running, Role: LEADER
I20260929 10:33:27.204226 20795 consensus_queue.cc:237] T 83bed19889b240828cb43f23578df343 P f56c1806329847c2aa32e3ef2fdcbaaf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f56c1806329847c2aa32e3ef2fdcbaaf" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 35589 } } peers { permanent_uuid: "6d38cd968e41446f908ccaf258e1045b" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 43181 } } peers { permanent_uuid: "734a32266f49438ba3d128f7528848eb" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 42985 } }
I20260929 10:33:27.207705 20319 catalog_manager.cc:5495] T 83bed19889b240828cb43f23578df343 P f56c1806329847c2aa32e3ef2fdcbaaf reported cstate change: term changed from 0 to 1, leader changed from <none> to f56c1806329847c2aa32e3ef2fdcbaaf (127.15.82.66). New cstate: current_term: 1 leader_uuid: "f56c1806329847c2aa32e3ef2fdcbaaf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f56c1806329847c2aa32e3ef2fdcbaaf" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 35589 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d38cd968e41446f908ccaf258e1045b" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 43181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "734a32266f49438ba3d128f7528848eb" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 42985 } health_report { overall_health: UNKNOWN } } }
I20260929 10:33:27.217990 20784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:27.216370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58158 (local address 127.15.82.66:35589)
0929 10:33:27.216510 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:27.216517 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:27.216549 (+ 32us) server_negotiation.cc:407] Connection header received
0929 10:33:27.216774 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:27.216776 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:27.216836 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:27.216931 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:27.217549 (+ 618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:27.217553 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:27.217556 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:33:27.217568 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:27.217595 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:33:27.217604 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:27.217695 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:27.217810 (+ 115us) server_negotiation.cc:299] Negotiation successful
0929 10:33:27.217850 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260929 10:33:27.223941 20725 raft_consensus.cc:1261] T 83bed19889b240828cb43f23578df343 P 6d38cd968e41446f908ccaf258e1045b [term 1 FOLLOWER]: Refusing update from remote peer f56c1806329847c2aa32e3ef2fdcbaaf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:33:27.223941 20463 raft_consensus.cc:1261] T 83bed19889b240828cb43f23578df343 P 734a32266f49438ba3d128f7528848eb [term 1 FOLLOWER]: Refusing update from remote peer f56c1806329847c2aa32e3ef2fdcbaaf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:33:27.224695 20798 consensus_queue.cc:1035] T 83bed19889b240828cb43f23578df343 P f56c1806329847c2aa32e3ef2fdcbaaf [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d38cd968e41446f908ccaf258e1045b" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 43181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:33:27.224924 20795 consensus_queue.cc:1035] T 83bed19889b240828cb43f23578df343 P f56c1806329847c2aa32e3ef2fdcbaaf [LEADER]: Connected to new peer: Peer: permanent_uuid: "734a32266f49438ba3d128f7528848eb" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 42985 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:33:27.231527 20801 mvcc.cc:205] Tried to move back new op lower bound from 7334617117581250560 to 7334617117510938624. Current Snapshot: MvccSnapshot[applied={T|T < 7334617117581250560}]
I20260929 10:33:27.232406 20803 mvcc.cc:205] Tried to move back new op lower bound from 7334617117581250560 to 7334617117510938624. Current Snapshot: MvccSnapshot[applied={T|T < 7334617117581250560}]
I20260929 10:33:27.241706 20319 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48902:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:33:27.241899 20319 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48902:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:33:27.244635 20319 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e2f04abb95c64590a0c9456f550837a4: Sending DeleteTablet for 3 replicas of tablet 83bed19889b240828cb43f23578df343
I20260929 10:33:27.245431 20574 tablet_service.cc:1514] Processing DeleteTablet for tablet 83bed19889b240828cb43f23578df343 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:33:27 UTC) from {username='slave'} at 127.0.0.1:58142
I20260929 10:33:27.245568 20705 tablet_service.cc:1514] Processing DeleteTablet for tablet 83bed19889b240828cb43f23578df343 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:33:27 UTC) from {username='slave'} at 127.0.0.1:38048
I20260929 10:33:27.245740 20443 tablet_service.cc:1514] Processing DeleteTablet for tablet 83bed19889b240828cb43f23578df343 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:33:27 UTC) from {username='slave'} at 127.0.0.1:52824
I20260929 10:33:27.246218 20813 tablet_replica.cc:330] T 83bed19889b240828cb43f23578df343 P 734a32266f49438ba3d128f7528848eb: stopping tablet replica
I20260929 10:33:27.246387 20811 tablet_replica.cc:330] T 83bed19889b240828cb43f23578df343 P f56c1806329847c2aa32e3ef2fdcbaaf: stopping tablet replica
I20260929 10:33:27.246620 20813 raft_consensus.cc:2229] T 83bed19889b240828cb43f23578df343 P 734a32266f49438ba3d128f7528848eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:33:27.246686 20811 raft_consensus.cc:2229] T 83bed19889b240828cb43f23578df343 P f56c1806329847c2aa32e3ef2fdcbaaf [term 1 LEADER]: Raft consensus shutting down.
I20260929 10:33:27.246966 20813 raft_consensus.cc:2258] T 83bed19889b240828cb43f23578df343 P 734a32266f49438ba3d128f7528848eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:33:27.247068 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 20381
I20260929 10:33:27.247715 20811 raft_consensus.cc:2258] T 83bed19889b240828cb43f23578df343 P f56c1806329847c2aa32e3ef2fdcbaaf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:33:27.249235 20811 ts_tablet_manager.cc:1905] T 83bed19889b240828cb43f23578df343 P f56c1806329847c2aa32e3ef2fdcbaaf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:33:27.250643 20812 tablet_replica.cc:330] T 83bed19889b240828cb43f23578df343 P 6d38cd968e41446f908ccaf258e1045b: stopping tablet replica
I20260929 10:33:27.251001 20812 raft_consensus.cc:2229] T 83bed19889b240828cb43f23578df343 P 6d38cd968e41446f908ccaf258e1045b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:33:27.251328 20812 raft_consensus.cc:2258] T 83bed19889b240828cb43f23578df343 P 6d38cd968e41446f908ccaf258e1045b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:33:27.252008 20811 ts_tablet_manager.cc:1918] T 83bed19889b240828cb43f23578df343 P f56c1806329847c2aa32e3ef2fdcbaaf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:33:27.252106 20811 log.cc:1196] T 83bed19889b240828cb43f23578df343 P f56c1806329847c2aa32e3ef2fdcbaaf: Deleting WAL directory at /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790677981592896-15689-0/minicluster-data/ts-1/wal/wals/83bed19889b240828cb43f23578df343
I20260929 10:33:27.252458 20811 ts_tablet_manager.cc:1939] T 83bed19889b240828cb43f23578df343 P f56c1806329847c2aa32e3ef2fdcbaaf: Deleting consensus metadata
I20260929 10:33:27.252914 20812 ts_tablet_manager.cc:1905] T 83bed19889b240828cb43f23578df343 P 6d38cd968e41446f908ccaf258e1045b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:33:27.253607 20307 catalog_manager.cc:4841] TS f56c1806329847c2aa32e3ef2fdcbaaf (127.15.82.66:35589): tablet 83bed19889b240828cb43f23578df343 (table test-table [id=5fce7a9d4a484a489a8482d76b8de6e8]) successfully deleted
W20260929 10:33:27.255743 20305 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.82.65:42985 (error 108)
I20260929 10:33:27.255949 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 20512
I20260929 10:33:27.257407 20782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:27.256525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.65:42985 (local address 127.0.0.1:52826)
0929 10:33:27.257207 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:27.257304 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.65:42985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":568}
W20260929 10:33:27.259675 20305 catalog_manager.cc:4584] TS 734a32266f49438ba3d128f7528848eb (127.15.82.65:42985): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 83bed19889b240828cb43f23578df343: Network error: Client connection negotiation failed: client connection to 127.15.82.65:42985: connect: Connection refused (error 111)
I20260929 10:33:27.259989 20812 ts_tablet_manager.cc:1918] T 83bed19889b240828cb43f23578df343 P 6d38cd968e41446f908ccaf258e1045b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:33:27.260110 20812 log.cc:1196] T 83bed19889b240828cb43f23578df343 P 6d38cd968e41446f908ccaf258e1045b: Deleting WAL directory at /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790677981592896-15689-0/minicluster-data/ts-2/wal/wals/83bed19889b240828cb43f23578df343
I20260929 10:33:27.260522 20812 ts_tablet_manager.cc:1939] T 83bed19889b240828cb43f23578df343 P 6d38cd968e41446f908ccaf258e1045b: Deleting consensus metadata
I20260929 10:33:27.261936 20305 catalog_manager.cc:4841] TS 6d38cd968e41446f908ccaf258e1045b (127.15.82.67:43181): tablet 83bed19889b240828cb43f23578df343 (table test-table [id=5fce7a9d4a484a489a8482d76b8de6e8]) successfully deleted
I20260929 10:33:27.262357 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 20643
I20260929 10:33:27.267834 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 20290
2026-09-29T10:33:27Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1010 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260929 10:33:27.289621 15689 mini_oidc.cc:152] Starting JWKS server
I20260929 10:33:27.291314 15689 webserver.cc:460] Webserver started at https://127.0.0.1:36893/ using document root <none> and password file <none>
I20260929 10:33:27.291505 15689 mini_oidc.cc:189] Starting OIDC Discovery server
I20260929 10:33:27.291785 15689 webserver.cc:460] Webserver started at http://127.0.0.1:44145/ using document root <none> and password file <none>
2026-09-29T10:33:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:33:27Z Disabled control of system clock
I20260929 10:33:27.300956 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790677981592896-15689-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790677981592896-15689-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790677981592896-15689-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:43367
--webserver_interface=127.15.82.126
--webserver_port=0
--builtin_ntp_servers=127.15.82.84:45809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.82.126:43367
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36893/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790677981592896-15689-0/testchainca.cert with env {}
W20260929 10:33:27.413486 20820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:33:27.413775 20820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:33:27.413816 20820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:33:27.417963 20820 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:33:27.418033 20820 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:33:27.418048 20820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:33:27.418083 20820 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:33:27.418098 20820 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260929 10:33:27.418113 20820 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260929 10:33:27.418133 20820 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36893/jwks.json
I20260929 10:33:27.423666 20820 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:45809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790677981592896-15689-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790677981592896-15689-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.82.126:43367
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:43367
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36893/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.82.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790677981592896-15689-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20820
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790677981592896-15689-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:33:27.424748 20820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:33:27.425783 20820 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:33:27.434347 20828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:27.434310 20825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:27.434332 20826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:27.434537 20820 server_base.cc:1034] running on GCE node
I20260929 10:33:27.435122 20820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:33:27.436177 20820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:33:27.437350 20820 hybrid_clock.cc:648] HybridClock initialized: now 1790678007437327 us; error 32 us; skew 500 ppm
I20260929 10:33:27.439302 20820 webserver.cc:460] Webserver started at http://127.15.82.126:44241/ using document root <none> and password file <none>
I20260929 10:33:27.439934 20820 fs_manager.cc:359] Metadata directory not provided
I20260929 10:33:27.440008 20820 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:33:27.440290 20820 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:33:27.442369 20820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790677981592896-15689-0/minicluster-data/master-0/data/instance:
uuid: "e37e6ccb5f3f49c783cf3155645bc2bc"
format_stamp: "Formatted at 2026-09-29 10:33:27 on dist-test-slave-vg89"
server_key: "dc59615043ea71ffc0520fc4bad08ba8"
server_key_iv: "a9fd2de4ce05d2477c95f9bb9d60d876"
server_key_version: "encryptionkey@0"
I20260929 10:33:27.443032 20820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790677981592896-15689-0/minicluster-data/master-0/wal/instance:
uuid: "e37e6ccb5f3f49c783cf3155645bc2bc"
format_stamp: "Formatted at 2026-09-29 10:33:27 on dist-test-slave-vg89"
server_key: "dc59615043ea71ffc0520fc4bad08ba8"
server_key_iv: "a9fd2de4ce05d2477c95f9bb9d60d876"
server_key_version: "encryptionkey@0"
I20260929 10:33:27.446739 20820 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260929 10:33:27.449168 20834 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:27.450495 20820 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 10:33:27.450640 20820 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790677981592896-15689-0/minicluster-data/master-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790677981592896-15689-0/minicluster-data/master-0/wal
uuid: "e37e6ccb5f3f49c783cf3155645bc2bc"
format_stamp: "Formatted at 2026-09-29 10:33:27 on dist-test-slave-vg89"
server_key: "dc59615043ea71ffc0520fc4bad08ba8"
server_key_iv: "a9fd2de4ce05d2477c95f9bb9d60d876"
server_key_version: "encryptionkey@0"
I20260929 10:33:27.450753 20820 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790677981592896-15689-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790677981592896-15689-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790677981592896-15689-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:33:27.465756 20820 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:33:27.466626 20820 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:33:27.466852 20820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:33:27.474641 20886 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.126:43367 every 8 connection(s)
I20260929 10:33:27.474643 20820 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.126:43367
I20260929 10:33:27.475739 20820 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
I20260929 10:33:27.476821 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 20820
I20260929 10:33:27.476987 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790677981592896-15689-0/minicluster-data/master-0/wal/instance
I20260929 10:33:27.477272 15689 external_mini_cluster.cc:1428] Setting key f6734b7a69c05bd5ea7825ee90faa182
I20260929 10:33:27.479463 20887 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:33:27.485236 20887 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e37e6ccb5f3f49c783cf3155645bc2bc: Bootstrap starting.
I20260929 10:33:27.486865 20890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:27.478797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59796 (local address 127.15.82.126:43367)
0929 10:33:27.479445 (+ 648us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:27.479470 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:27.479533 (+ 63us) server_negotiation.cc:407] Connection header received
0929 10:33:27.480235 (+ 702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:27.480253 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:27.480664 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:27.481037 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:27.481470 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:27.482520 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:27.483450 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:27.483931 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:27.484107 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:27.485005 (+ 898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:27.485009 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:27.485023 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:33:27.485469 (+ 446us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:27.485524 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:33:27.485546 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:27.485662 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:27.485812 (+ 150us) server_negotiation.cc:299] Negotiation successful
0929 10:33:27.486073 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":137,"threads_started":1}
I20260929 10:33:27.487754 20887 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e37e6ccb5f3f49c783cf3155645bc2bc: Neither blocks nor log segments found. Creating new log.
I20260929 10:33:27.488517 20887 log.cc:824] T 00000000000000000000000000000000 P e37e6ccb5f3f49c783cf3155645bc2bc: Log is configured to *not* fsync() on all Append() calls
I20260929 10:33:27.489740 15689 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 10:33:27.490866 20887 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e37e6ccb5f3f49c783cf3155645bc2bc: No bootstrap required, opened a new log
I20260929 10:33:27.494638 20887 raft_consensus.cc:348] T 00000000000000000000000000000000 P e37e6ccb5f3f49c783cf3155645bc2bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e37e6ccb5f3f49c783cf3155645bc2bc" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 43367 } }
I20260929 10:33:27.494867 20887 raft_consensus.cc:374] T 00000000000000000000000000000000 P e37e6ccb5f3f49c783cf3155645bc2bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:33:27.494902 20887 raft_consensus.cc:729] T 00000000000000000000000000000000 P e37e6ccb5f3f49c783cf3155645bc2bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e37e6ccb5f3f49c783cf3155645bc2bc, State: Initialized, Role: FOLLOWER
I20260929 10:33:27.495354 20887 consensus_queue.cc:260] T 00000000000000000000000000000000 P e37e6ccb5f3f49c783cf3155645bc2bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e37e6ccb5f3f49c783cf3155645bc2bc" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 43367 } }
I20260929 10:33:27.495471 20887 raft_consensus.cc:388] T 00000000000000000000000000000000 P e37e6ccb5f3f49c783cf3155645bc2bc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:33:27.495558 20887 raft_consensus.cc:482] T 00000000000000000000000000000000 P e37e6ccb5f3f49c783cf3155645bc2bc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:33:27.495671 20887 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e37e6ccb5f3f49c783cf3155645bc2bc [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:33:27.496547 20887 raft_consensus.cc:504] T 00000000000000000000000000000000 P e37e6ccb5f3f49c783cf3155645bc2bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e37e6ccb5f3f49c783cf3155645bc2bc" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 43367 } }
I20260929 10:33:27.496887 20887 leader_election.cc:302] T 00000000000000000000000000000000 P e37e6ccb5f3f49c783cf3155645bc2bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e37e6ccb5f3f49c783cf3155645bc2bc; no voters:
I20260929 10:33:27.497236 20887 leader_election.cc:288] T 00000000000000000000000000000000 P e37e6ccb5f3f49c783cf3155645bc2bc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:33:27.497359 20892 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e37e6ccb5f3f49c783cf3155645bc2bc [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:33:27.497560 20892 raft_consensus.cc:686] T 00000000000000000000000000000000 P e37e6ccb5f3f49c783cf3155645bc2bc [term 1 LEADER]: Becoming Leader. State: Replica: e37e6ccb5f3f49c783cf3155645bc2bc, State: Running, Role: LEADER
I20260929 10:33:27.497845 20892 consensus_queue.cc:237] T 00000000000000000000000000000000 P e37e6ccb5f3f49c783cf3155645bc2bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e37e6ccb5f3f49c783cf3155645bc2bc" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 43367 } }
I20260929 10:33:27.498555 20887 sys_catalog.cc:564] T 00000000000000000000000000000000 P e37e6ccb5f3f49c783cf3155645bc2bc [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:33:27.499557 20894 sys_catalog.cc:455] T 00000000000000000000000000000000 P e37e6ccb5f3f49c783cf3155645bc2bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader e37e6ccb5f3f49c783cf3155645bc2bc. Latest consensus state: current_term: 1 leader_uuid: "e37e6ccb5f3f49c783cf3155645bc2bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e37e6ccb5f3f49c783cf3155645bc2bc" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 43367 } } }
I20260929 10:33:27.499683 20894 sys_catalog.cc:458] T 00000000000000000000000000000000 P e37e6ccb5f3f49c783cf3155645bc2bc [sys.catalog]: This master's current role is: LEADER
I20260929 10:33:27.500095 20901 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:33:27.500082 20893 sys_catalog.cc:455] T 00000000000000000000000000000000 P e37e6ccb5f3f49c783cf3155645bc2bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e37e6ccb5f3f49c783cf3155645bc2bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e37e6ccb5f3f49c783cf3155645bc2bc" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 43367 } } }
I20260929 10:33:27.500298 20893 sys_catalog.cc:458] T 00000000000000000000000000000000 P e37e6ccb5f3f49c783cf3155645bc2bc [sys.catalog]: This master's current role is: LEADER
I20260929 10:33:27.503271 20901 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:33:27.508227 20901 catalog_manager.cc:1347] Generated new cluster ID: d096dec32a204260a4fda09ca6e42b30
I20260929 10:33:27.508302 20901 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:33:27.520690 20901 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:33:27.521883 20901 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:33:27.538666 20901 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e37e6ccb5f3f49c783cf3155645bc2bc: Generated new TSK 0
I20260929 10:33:27.539569 20901 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260929 10:33:27.575225 20917 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260929 10:33:27.575346 20917 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260929 10:33:27.575814 20890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:27.550599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59800 (local address 127.15.82.126:43367)
0929 10:33:27.550737 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:27.550742 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:27.551056 (+ 314us) server_negotiation.cc:407] Connection header received
0929 10:33:27.551283 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:27.551287 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:27.551359 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:27.551481 (+ 122us) server_negotiation.cc:226] Negotiated authn=JWT
0929 10:33:27.551777 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:27.552592 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:27.553824 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:27.554212 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:27.554351 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:27.554770 (+ 419us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0929 10:33:27.575418 (+ 20648us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36893/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0929 10:33:27.575690 (+ 272us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59800: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36893/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":52}
W20260929 10:33:27.575901 20890 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59800: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36893/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260929 10:33:27.577267 20916 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:33:27.550554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:43367 (local address 127.0.0.1:59800)
0929 10:33:27.550946 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:27.550961 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:27.551096 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:27.551506 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:27.551509 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:27.551518 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0929 10:33:27.551641 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:27.551647 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:27.552704 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:27.552708 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:27.553697 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:27.553707 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:27.554378 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:27.554381 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:27.554497 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:27.554551 (+ 54us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0929 10:33:27.575690 (+ 21139us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36893/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0929 10:33:27.575941 (+ 251us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.15.82.126:43367: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36893/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":125,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.15.82.126:43367: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36893/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260929 10:33:27.577996 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 20820
2026-09-29T10:33:27Z chronyd exiting
I20260929 10:33:27.615203 15689 test_util.cc:175] -----------------------------------------------
I20260929 10:33:27.615296 15689 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790677981592896-15689-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (327 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260929 10:33:27.616335 15689 mini_oidc.cc:152] Starting JWKS server
I20260929 10:33:27.617199 15689 webserver.cc:460] Webserver started at https://127.0.0.1:40653/ using document root <none> and password file <none>
I20260929 10:33:27.617311 15689 mini_oidc.cc:189] Starting OIDC Discovery server
I20260929 10:33:27.617616 15689 webserver.cc:460] Webserver started at http://127.0.0.1:42165/ using document root <none> and password file <none>
2026-09-29T10:33:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:33:27Z Disabled control of system clock
I20260929 10:33:27.626842 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790677981592896-15689-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790677981592896-15689-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790677981592896-15689-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:43151
--webserver_interface=127.15.82.126
--webserver_port=0
--builtin_ntp_servers=127.15.82.84:34607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.82.126:43151
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40653/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790677981592896-15689-0/testchainca.cert with env {}
W20260929 10:33:27.735615 20923 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:33:27.735960 20923 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:33:27.736043 20923 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:33:27.740131 20923 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:33:27.740231 20923 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:33:27.740260 20923 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:33:27.740291 20923 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:33:27.740330 20923 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260929 10:33:27.740366 20923 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260929 10:33:27.740391 20923 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40653/jwks.json
I20260929 10:33:27.745869 20923 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:34607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790677981592896-15689-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790677981592896-15689-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.82.126:43151
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:43151
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40653/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.82.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790677981592896-15689-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20923
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790677981592896-15689-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:33:27.747016 20923 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:33:27.748049 20923 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:33:27.756443 20929 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:27.756453 20928 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:27.756441 20931 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:27.756666 20923 server_base.cc:1034] running on GCE node
I20260929 10:33:27.757418 20923 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:33:27.758571 20923 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:33:27.759732 20923 hybrid_clock.cc:648] HybridClock initialized: now 1790678007759718 us; error 25 us; skew 500 ppm
I20260929 10:33:27.761533 20923 webserver.cc:460] Webserver started at http://127.15.82.126:40841/ using document root <none> and password file <none>
I20260929 10:33:27.762178 20923 fs_manager.cc:359] Metadata directory not provided
I20260929 10:33:27.762247 20923 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:33:27.762512 20923 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:33:27.764567 20923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790677981592896-15689-0/minicluster-data/master-0/data/instance:
uuid: "e2278410d5a147b1b585498d21ad611a"
format_stamp: "Formatted at 2026-09-29 10:33:27 on dist-test-slave-vg89"
server_key: "4b7a4445228c84afb5589a098df02d4f"
server_key_iv: "6e1ac305fce26927787cf9021834de0c"
server_key_version: "encryptionkey@0"
I20260929 10:33:27.765116 20923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790677981592896-15689-0/minicluster-data/master-0/wal/instance:
uuid: "e2278410d5a147b1b585498d21ad611a"
format_stamp: "Formatted at 2026-09-29 10:33:27 on dist-test-slave-vg89"
server_key: "4b7a4445228c84afb5589a098df02d4f"
server_key_iv: "6e1ac305fce26927787cf9021834de0c"
server_key_version: "encryptionkey@0"
I20260929 10:33:27.768790 20923 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260929 10:33:27.771193 20937 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:27.772471 20923 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260929 10:33:27.772614 20923 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790677981592896-15689-0/minicluster-data/master-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790677981592896-15689-0/minicluster-data/master-0/wal
uuid: "e2278410d5a147b1b585498d21ad611a"
format_stamp: "Formatted at 2026-09-29 10:33:27 on dist-test-slave-vg89"
server_key: "4b7a4445228c84afb5589a098df02d4f"
server_key_iv: "6e1ac305fce26927787cf9021834de0c"
server_key_version: "encryptionkey@0"
I20260929 10:33:27.772725 20923 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790677981592896-15689-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790677981592896-15689-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790677981592896-15689-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:33:27.791270 20923 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:33:27.791981 20923 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:33:27.792181 20923 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:33:27.799132 20923 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.126:43151
I20260929 10:33:27.799161 20989 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.126:43151 every 8 connection(s)
I20260929 10:33:27.800091 20923 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
I20260929 10:33:27.802721 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 20923
I20260929 10:33:27.802855 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790677981592896-15689-0/minicluster-data/master-0/wal/instance
I20260929 10:33:27.803169 15689 external_mini_cluster.cc:1428] Setting key 61506e6f08a6ae859f72b023a7da0765
I20260929 10:33:27.803571 20990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:33:27.809738 20990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2278410d5a147b1b585498d21ad611a: Bootstrap starting.
I20260929 10:33:27.812284 20990 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e2278410d5a147b1b585498d21ad611a: Neither blocks nor log segments found. Creating new log.
I20260929 10:33:27.812628 20993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:27.804564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45340 (local address 127.15.82.126:43151)
0929 10:33:27.805208 (+ 644us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:27.805226 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:27.805261 (+ 35us) server_negotiation.cc:407] Connection header received
0929 10:33:27.806288 (+ 1027us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:27.806306 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:27.806755 (+ 449us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:27.807081 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:27.807489 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:27.808457 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:27.809360 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:27.809827 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:27.809993 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:27.810899 (+ 906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:27.810904 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:27.810915 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:33:27.811283 (+ 368us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:27.811337 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:33:27.811356 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:27.811431 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:27.811592 (+ 161us) server_negotiation.cc:299] Negotiation successful
0929 10:33:27.811880 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":152,"threads_started":1}
I20260929 10:33:27.813114 20990 log.cc:824] T 00000000000000000000000000000000 P e2278410d5a147b1b585498d21ad611a: Log is configured to *not* fsync() on all Append() calls
I20260929 10:33:27.815048 20990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2278410d5a147b1b585498d21ad611a: No bootstrap required, opened a new log
I20260929 10:33:27.815312 15689 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 10:33:27.818802 20990 raft_consensus.cc:348] T 00000000000000000000000000000000 P e2278410d5a147b1b585498d21ad611a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2278410d5a147b1b585498d21ad611a" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 43151 } }
I20260929 10:33:27.819020 20990 raft_consensus.cc:374] T 00000000000000000000000000000000 P e2278410d5a147b1b585498d21ad611a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:33:27.819054 20990 raft_consensus.cc:729] T 00000000000000000000000000000000 P e2278410d5a147b1b585498d21ad611a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2278410d5a147b1b585498d21ad611a, State: Initialized, Role: FOLLOWER
I20260929 10:33:27.819465 20990 consensus_queue.cc:260] T 00000000000000000000000000000000 P e2278410d5a147b1b585498d21ad611a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2278410d5a147b1b585498d21ad611a" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 43151 } }
I20260929 10:33:27.819594 20990 raft_consensus.cc:388] T 00000000000000000000000000000000 P e2278410d5a147b1b585498d21ad611a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:33:27.819641 20990 raft_consensus.cc:482] T 00000000000000000000000000000000 P e2278410d5a147b1b585498d21ad611a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:33:27.819711 20990 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e2278410d5a147b1b585498d21ad611a [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:33:27.820554 20990 raft_consensus.cc:504] T 00000000000000000000000000000000 P e2278410d5a147b1b585498d21ad611a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2278410d5a147b1b585498d21ad611a" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 43151 } }
I20260929 10:33:27.820847 20990 leader_election.cc:302] T 00000000000000000000000000000000 P e2278410d5a147b1b585498d21ad611a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e2278410d5a147b1b585498d21ad611a; no voters:
I20260929 10:33:27.821092 20990 leader_election.cc:288] T 00000000000000000000000000000000 P e2278410d5a147b1b585498d21ad611a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:33:27.821233 20995 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e2278410d5a147b1b585498d21ad611a [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:33:27.821470 20995 raft_consensus.cc:686] T 00000000000000000000000000000000 P e2278410d5a147b1b585498d21ad611a [term 1 LEADER]: Becoming Leader. State: Replica: e2278410d5a147b1b585498d21ad611a, State: Running, Role: LEADER
I20260929 10:33:27.821765 20995 consensus_queue.cc:237] T 00000000000000000000000000000000 P e2278410d5a147b1b585498d21ad611a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2278410d5a147b1b585498d21ad611a" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 43151 } }
I20260929 10:33:27.822321 20990 sys_catalog.cc:564] T 00000000000000000000000000000000 P e2278410d5a147b1b585498d21ad611a [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:33:27.823436 20996 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2278410d5a147b1b585498d21ad611a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e2278410d5a147b1b585498d21ad611a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2278410d5a147b1b585498d21ad611a" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 43151 } } }
I20260929 10:33:27.823577 20997 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2278410d5a147b1b585498d21ad611a [sys.catalog]: SysCatalogTable state changed. Reason: New leader e2278410d5a147b1b585498d21ad611a. Latest consensus state: current_term: 1 leader_uuid: "e2278410d5a147b1b585498d21ad611a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2278410d5a147b1b585498d21ad611a" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 43151 } } }
I20260929 10:33:27.823659 20997 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2278410d5a147b1b585498d21ad611a [sys.catalog]: This master's current role is: LEADER
I20260929 10:33:27.823626 20996 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2278410d5a147b1b585498d21ad611a [sys.catalog]: This master's current role is: LEADER
I20260929 10:33:27.824026 21004 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:33:27.826906 21004 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:33:27.831928 21004 catalog_manager.cc:1347] Generated new cluster ID: 81f7425b19574fc6ba9f21dd372fd3c7
I20260929 10:33:27.831995 21004 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:33:27.840003 21004 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:33:27.841189 21004 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:33:27.855773 21004 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e2278410d5a147b1b585498d21ad611a: Generated new TSK 0
I20260929 10:33:27.856608 21004 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260929 10:33:27.897505 21020 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260929 10:33:27.897589 21020 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260929 10:33:27.898188 20993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:27.875014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45346 (local address 127.15.82.126:43151)
0929 10:33:27.875148 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:27.875152 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:27.875263 (+ 111us) server_negotiation.cc:407] Connection header received
0929 10:33:27.875428 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:27.875430 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:27.875473 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:27.875560 (+ 87us) server_negotiation.cc:226] Negotiated authn=JWT
0929 10:33:27.875826 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:27.876508 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:27.877756 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:27.877958 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:27.878037 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:27.878374 (+ 337us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0929 10:33:27.897855 (+ 19481us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40653/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0929 10:33:27.898070 (+ 215us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45346: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40653/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":56}
W20260929 10:33:27.898272 20993 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45346: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40653/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260929 10:33:27.898432 21019 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:33:27.874855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:43151 (local address 127.0.0.1:45346)
0929 10:33:27.875171 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:27.875186 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:27.875309 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:27.875588 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:27.875591 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:27.875599 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0929 10:33:27.875719 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:27.875728 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:27.876610 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:27.876614 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:27.877637 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:27.877648 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:27.878083 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:27.878087 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:27.878177 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:27.878206 (+ 29us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0929 10:33:27.898039 (+ 19833us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40653/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0929 10:33:27.898220 (+ 181us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.15.82.126:43151: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40653/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":100,"threads_started":1}
I20260929 10:33:27.899127 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 20923
2026-09-29T10:33:27Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (322 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260929 10:33:27.938748 15689 mini_oidc.cc:152] Starting JWKS server
I20260929 10:33:27.939571 15689 webserver.cc:460] Webserver started at https://127.0.0.1:43453/ using document root <none> and password file <none>
I20260929 10:33:27.939653 15689 mini_oidc.cc:189] Starting OIDC Discovery server
I20260929 10:33:27.939886 15689 webserver.cc:460] Webserver started at http://127.0.0.1:33007/ using document root <none> and password file <none>
2026-09-29T10:33:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:33:27Z Disabled control of system clock
I20260929 10:33:27.949115 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790677981592896-15689-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790677981592896-15689-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790677981592896-15689-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:37699
--webserver_interface=127.15.82.126
--webserver_port=0
--builtin_ntp_servers=127.15.82.84:37119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.82.126:37699
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43453/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260929 10:33:28.059020 21026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:33:28.059319 21026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:33:28.059358 21026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:33:28.063287 21026 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:33:28.063347 21026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:33:28.063362 21026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:33:28.063380 21026 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:33:28.063395 21026 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260929 10:33:28.063411 21026 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260929 10:33:28.063424 21026 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43453/jwks.json
I20260929 10:33:28.068274 21026 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:37119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790677981592896-15689-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790677981592896-15689-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.82.126:37699
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:37699
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43453/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.82.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21026
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790677981592896-15689-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:33:28.069376 21026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:33:28.070868 21026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:33:28.079783 21034 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:28.079945 21026 server_base.cc:1034] running on GCE node
W20260929 10:33:28.079957 21032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:28.079788 21031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:28.080790 21026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:33:28.081975 21026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:33:28.083165 21026 hybrid_clock.cc:648] HybridClock initialized: now 1790678008083147 us; error 29 us; skew 500 ppm
I20260929 10:33:28.085203 21026 webserver.cc:460] Webserver started at http://127.15.82.126:42181/ using document root <none> and password file <none>
I20260929 10:33:28.085852 21026 fs_manager.cc:359] Metadata directory not provided
I20260929 10:33:28.085954 21026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:33:28.086206 21026 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:33:28.088454 21026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790677981592896-15689-0/minicluster-data/master-0/data/instance:
uuid: "bbdb32dcb19a4f0398390433f8f23415"
format_stamp: "Formatted at 2026-09-29 10:33:28 on dist-test-slave-vg89"
server_key: "bfc89f7e1b33b67ca0e68febfd29b919"
server_key_iv: "852593dd55bd3b31b4356580ee8dc045"
server_key_version: "encryptionkey@0"
I20260929 10:33:28.089059 21026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790677981592896-15689-0/minicluster-data/master-0/wal/instance:
uuid: "bbdb32dcb19a4f0398390433f8f23415"
format_stamp: "Formatted at 2026-09-29 10:33:28 on dist-test-slave-vg89"
server_key: "bfc89f7e1b33b67ca0e68febfd29b919"
server_key_iv: "852593dd55bd3b31b4356580ee8dc045"
server_key_version: "encryptionkey@0"
I20260929 10:33:28.092988 21026 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260929 10:33:28.095463 21040 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:28.096751 21026 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 10:33:28.096899 21026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790677981592896-15689-0/minicluster-data/master-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790677981592896-15689-0/minicluster-data/master-0/wal
uuid: "bbdb32dcb19a4f0398390433f8f23415"
format_stamp: "Formatted at 2026-09-29 10:33:28 on dist-test-slave-vg89"
server_key: "bfc89f7e1b33b67ca0e68febfd29b919"
server_key_iv: "852593dd55bd3b31b4356580ee8dc045"
server_key_version: "encryptionkey@0"
I20260929 10:33:28.097023 21026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790677981592896-15689-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790677981592896-15689-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790677981592896-15689-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:33:28.117555 21026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:33:28.118562 21026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:33:28.118727 21026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:33:28.125910 21026 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.126:37699
I20260929 10:33:28.125991 21092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.126:37699 every 8 connection(s)
I20260929 10:33:28.127027 21026 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
I20260929 10:33:28.130349 21093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:33:28.135090 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 21026
I20260929 10:33:28.135196 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790677981592896-15689-0/minicluster-data/master-0/wal/instance
I20260929 10:33:28.135478 15689 external_mini_cluster.cc:1428] Setting key 95e2b55431199c568acca5c1d7039333
I20260929 10:33:28.136282 21093 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbdb32dcb19a4f0398390433f8f23415: Bootstrap starting.
I20260929 10:33:28.138578 21093 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bbdb32dcb19a4f0398390433f8f23415: Neither blocks nor log segments found. Creating new log.
I20260929 10:33:28.139470 21093 log.cc:824] T 00000000000000000000000000000000 P bbdb32dcb19a4f0398390433f8f23415: Log is configured to *not* fsync() on all Append() calls
I20260929 10:33:28.141726 21093 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbdb32dcb19a4f0398390433f8f23415: No bootstrap required, opened a new log
I20260929 10:33:28.145053 21096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:28.136890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36674 (local address 127.15.82.126:37699)
0929 10:33:28.137528 (+ 638us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:28.137545 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:28.137580 (+ 35us) server_negotiation.cc:407] Connection header received
0929 10:33:28.138205 (+ 625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:28.138221 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:28.138706 (+ 485us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:28.139016 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:28.139376 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:28.140646 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:28.141836 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:28.142315 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:28.142498 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:28.143307 (+ 809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:28.143312 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:28.143324 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:33:28.143701 (+ 377us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:28.143767 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:33:28.143787 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:28.143861 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:28.144027 (+ 166us) server_negotiation.cc:299] Negotiation successful
0929 10:33:28.144321 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":139,"threads_started":1}
I20260929 10:33:28.145706 21093 raft_consensus.cc:348] T 00000000000000000000000000000000 P bbdb32dcb19a4f0398390433f8f23415 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbdb32dcb19a4f0398390433f8f23415" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 37699 } }
I20260929 10:33:28.145961 21093 raft_consensus.cc:374] T 00000000000000000000000000000000 P bbdb32dcb19a4f0398390433f8f23415 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:33:28.146039 21093 raft_consensus.cc:729] T 00000000000000000000000000000000 P bbdb32dcb19a4f0398390433f8f23415 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbdb32dcb19a4f0398390433f8f23415, State: Initialized, Role: FOLLOWER
I20260929 10:33:28.146524 21093 consensus_queue.cc:260] T 00000000000000000000000000000000 P bbdb32dcb19a4f0398390433f8f23415 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbdb32dcb19a4f0398390433f8f23415" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 37699 } }
I20260929 10:33:28.146682 21093 raft_consensus.cc:388] T 00000000000000000000000000000000 P bbdb32dcb19a4f0398390433f8f23415 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:33:28.146736 21093 raft_consensus.cc:482] T 00000000000000000000000000000000 P bbdb32dcb19a4f0398390433f8f23415 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:33:28.146819 21093 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bbdb32dcb19a4f0398390433f8f23415 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:33:28.147933 21093 raft_consensus.cc:504] T 00000000000000000000000000000000 P bbdb32dcb19a4f0398390433f8f23415 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbdb32dcb19a4f0398390433f8f23415" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 37699 } }
I20260929 10:33:28.148064 15689 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 10:33:28.148306 21093 leader_election.cc:302] T 00000000000000000000000000000000 P bbdb32dcb19a4f0398390433f8f23415 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bbdb32dcb19a4f0398390433f8f23415; no voters:
I20260929 10:33:28.148643 21093 leader_election.cc:288] T 00000000000000000000000000000000 P bbdb32dcb19a4f0398390433f8f23415 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:33:28.148756 21098 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bbdb32dcb19a4f0398390433f8f23415 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:33:28.148994 21098 raft_consensus.cc:686] T 00000000000000000000000000000000 P bbdb32dcb19a4f0398390433f8f23415 [term 1 LEADER]: Becoming Leader. State: Replica: bbdb32dcb19a4f0398390433f8f23415, State: Running, Role: LEADER
I20260929 10:33:28.149322 21098 consensus_queue.cc:237] T 00000000000000000000000000000000 P bbdb32dcb19a4f0398390433f8f23415 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbdb32dcb19a4f0398390433f8f23415" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 37699 } }
I20260929 10:33:28.149991 21093 sys_catalog.cc:564] T 00000000000000000000000000000000 P bbdb32dcb19a4f0398390433f8f23415 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:33:28.151191 21099 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbdb32dcb19a4f0398390433f8f23415 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bbdb32dcb19a4f0398390433f8f23415" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbdb32dcb19a4f0398390433f8f23415" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 37699 } } }
I20260929 10:33:28.151192 21100 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbdb32dcb19a4f0398390433f8f23415 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bbdb32dcb19a4f0398390433f8f23415. Latest consensus state: current_term: 1 leader_uuid: "bbdb32dcb19a4f0398390433f8f23415" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbdb32dcb19a4f0398390433f8f23415" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 37699 } } }
I20260929 10:33:28.151371 21100 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbdb32dcb19a4f0398390433f8f23415 [sys.catalog]: This master's current role is: LEADER
I20260929 10:33:28.151371 21099 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbdb32dcb19a4f0398390433f8f23415 [sys.catalog]: This master's current role is: LEADER
I20260929 10:33:28.151798 21107 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:33:28.155031 21107 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:33:28.160637 21107 catalog_manager.cc:1347] Generated new cluster ID: 17af10add03245de9af5aa8e71857436
I20260929 10:33:28.160705 21107 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:33:28.181574 21107 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:33:28.182621 21107 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:33:28.187618 21107 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bbdb32dcb19a4f0398390433f8f23415: Generated new TSK 0
I20260929 10:33:28.188381 21107 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 10:33:28.213289 21096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:28.207651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36676 (local address 127.15.82.126:37699)
0929 10:33:28.207810 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:28.207815 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:28.207999 (+ 184us) server_negotiation.cc:407] Connection header received
0929 10:33:28.208177 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:28.208180 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:28.208229 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:28.208325 (+ 96us) server_negotiation.cc:226] Negotiated authn=JWT
0929 10:33:28.208573 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:28.209121 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:28.210334 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:28.210623 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:28.210700 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:28.211029 (+ 329us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0929 10:33:28.212877 (+ 1848us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43453/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0929 10:33:28.213136 (+ 259us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36676: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43453/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":57}
W20260929 10:33:28.213371 21122 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:33:28.207586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:37699 (local address 127.0.0.1:36676)
0929 10:33:28.207905 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:28.207920 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:28.208038 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:28.208345 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:28.208348 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:28.208356 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0929 10:33:28.208471 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:28.208477 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:28.209228 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:28.209232 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:28.210214 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:28.210224 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:28.210737 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:28.210740 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:28.210832 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:28.210861 (+ 29us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0929 10:33:28.213042 (+ 2181us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43453/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0929 10:33:28.213182 (+ 140us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.15.82.126:37699: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43453/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":124,"threads_started":1}
W20260929 10:33:28.213378 21096 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36676: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43453/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
I20260929 10:33:28.213975 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 21026
E20260929 10:33:28.224748 21123 webserver.cc:533] Webserver: sslize failed:
E20260929 10:33:28.224845 21123 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-09-29T10:33:28Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (314 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-29T10:33:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:33:28Z Disabled control of system clock
I20260929 10:33:28.262235 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790677981592896-15689-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790677981592896-15689-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790677981592896-15689-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:46661
--webserver_interface=127.15.82.126
--webserver_port=0
--builtin_ntp_servers=127.15.82.84:33965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.82.126:46661
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790677981592896-15689-0/testchainca.cert with env {}
W20260929 10:33:28.372799 21127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:33:28.373104 21127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:33:28.373176 21127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:33:28.377257 21127 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:33:28.377343 21127 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:33:28.377372 21127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:33:28.377409 21127 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:33:28.377440 21127 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260929 10:33:28.377475 21127 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260929 10:33:28.377509 21127 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260929 10:33:28.382861 21127 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:33965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790677981592896-15689-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790677981592896-15689-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.82.126:46661
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:46661
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.82.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790677981592896-15689-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21127
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790677981592896-15689-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:33:28.383966 21127 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:33:28.385012 21127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:33:28.393661 21132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:28.393785 21127 server_base.cc:1034] running on GCE node
W20260929 10:33:28.393719 21135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:28.393702 21133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:28.394526 21127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:33:28.395587 21127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:33:28.396759 21127 hybrid_clock.cc:648] HybridClock initialized: now 1790678008396744 us; error 25 us; skew 500 ppm
I20260929 10:33:28.398658 21127 webserver.cc:460] Webserver started at http://127.15.82.126:35139/ using document root <none> and password file <none>
I20260929 10:33:28.399238 21127 fs_manager.cc:359] Metadata directory not provided
I20260929 10:33:28.399300 21127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:33:28.399514 21127 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:33:28.401594 21127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790677981592896-15689-0/minicluster-data/master-0/data/instance:
uuid: "b530997f2a5e4ea295a2b71b70610b76"
format_stamp: "Formatted at 2026-09-29 10:33:28 on dist-test-slave-vg89"
server_key: "e24cc8b3e9528e87874fc6a0ad23dccb"
server_key_iv: "b87b448e702b119b071ad5434433f53b"
server_key_version: "encryptionkey@0"
I20260929 10:33:28.402184 21127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790677981592896-15689-0/minicluster-data/master-0/wal/instance:
uuid: "b530997f2a5e4ea295a2b71b70610b76"
format_stamp: "Formatted at 2026-09-29 10:33:28 on dist-test-slave-vg89"
server_key: "e24cc8b3e9528e87874fc6a0ad23dccb"
server_key_iv: "b87b448e702b119b071ad5434433f53b"
server_key_version: "encryptionkey@0"
I20260929 10:33:28.405941 21127 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260929 10:33:28.408427 21141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:28.409626 21127 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 10:33:28.409780 21127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790677981592896-15689-0/minicluster-data/master-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790677981592896-15689-0/minicluster-data/master-0/wal
uuid: "b530997f2a5e4ea295a2b71b70610b76"
format_stamp: "Formatted at 2026-09-29 10:33:28 on dist-test-slave-vg89"
server_key: "e24cc8b3e9528e87874fc6a0ad23dccb"
server_key_iv: "b87b448e702b119b071ad5434433f53b"
server_key_version: "encryptionkey@0"
I20260929 10:33:28.409899 21127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790677981592896-15689-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790677981592896-15689-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790677981592896-15689-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:33:28.436344 21127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:33:28.437072 21127 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:33:28.437278 21127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:33:28.444129 21127 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.126:46661
I20260929 10:33:28.444159 21193 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.126:46661 every 8 connection(s)
I20260929 10:33:28.445102 21127 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
I20260929 10:33:28.448004 21194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:33:28.448345 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 21127
I20260929 10:33:28.448463 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790677981592896-15689-0/minicluster-data/master-0/wal/instance
I20260929 10:33:28.448777 15689 external_mini_cluster.cc:1428] Setting key c866e299c378a4adad65ec8a8709f6e1
I20260929 10:33:28.454667 21194 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b530997f2a5e4ea295a2b71b70610b76: Bootstrap starting.
I20260929 10:33:28.457013 21194 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b530997f2a5e4ea295a2b71b70610b76: Neither blocks nor log segments found. Creating new log.
I20260929 10:33:28.457818 21194 log.cc:824] T 00000000000000000000000000000000 P b530997f2a5e4ea295a2b71b70610b76: Log is configured to *not* fsync() on all Append() calls
I20260929 10:33:28.458127 21197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:28.450108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43990 (local address 127.15.82.126:46661)
0929 10:33:28.450816 (+ 708us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:28.450831 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:28.450869 (+ 38us) server_negotiation.cc:407] Connection header received
0929 10:33:28.451537 (+ 668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:28.451555 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:28.451953 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:28.452317 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:28.452771 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:28.453732 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:28.454710 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:28.455252 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:28.455429 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:28.456332 (+ 903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:28.456337 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:28.456350 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:33:28.456739 (+ 389us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:28.456802 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:33:28.456821 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:28.456938 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:28.457099 (+ 161us) server_negotiation.cc:299] Negotiation successful
0929 10:33:28.457364 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":128,"threads_started":1}
I20260929 10:33:28.459910 21194 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b530997f2a5e4ea295a2b71b70610b76: No bootstrap required, opened a new log
I20260929 10:33:28.460961 15689 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 10:33:28.463882 21194 raft_consensus.cc:348] T 00000000000000000000000000000000 P b530997f2a5e4ea295a2b71b70610b76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b530997f2a5e4ea295a2b71b70610b76" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 46661 } }
I20260929 10:33:28.464131 21194 raft_consensus.cc:374] T 00000000000000000000000000000000 P b530997f2a5e4ea295a2b71b70610b76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:33:28.464218 21194 raft_consensus.cc:729] T 00000000000000000000000000000000 P b530997f2a5e4ea295a2b71b70610b76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b530997f2a5e4ea295a2b71b70610b76, State: Initialized, Role: FOLLOWER
I20260929 10:33:28.464651 21194 consensus_queue.cc:260] T 00000000000000000000000000000000 P b530997f2a5e4ea295a2b71b70610b76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b530997f2a5e4ea295a2b71b70610b76" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 46661 } }
I20260929 10:33:28.464800 21194 raft_consensus.cc:388] T 00000000000000000000000000000000 P b530997f2a5e4ea295a2b71b70610b76 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:33:28.464887 21194 raft_consensus.cc:482] T 00000000000000000000000000000000 P b530997f2a5e4ea295a2b71b70610b76 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:33:28.465005 21194 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b530997f2a5e4ea295a2b71b70610b76 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:33:28.465943 21194 raft_consensus.cc:504] T 00000000000000000000000000000000 P b530997f2a5e4ea295a2b71b70610b76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b530997f2a5e4ea295a2b71b70610b76" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 46661 } }
I20260929 10:33:28.466288 21194 leader_election.cc:302] T 00000000000000000000000000000000 P b530997f2a5e4ea295a2b71b70610b76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b530997f2a5e4ea295a2b71b70610b76; no voters:
I20260929 10:33:28.466683 21194 leader_election.cc:288] T 00000000000000000000000000000000 P b530997f2a5e4ea295a2b71b70610b76 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:33:28.466807 21199 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b530997f2a5e4ea295a2b71b70610b76 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:33:28.467060 21199 raft_consensus.cc:686] T 00000000000000000000000000000000 P b530997f2a5e4ea295a2b71b70610b76 [term 1 LEADER]: Becoming Leader. State: Replica: b530997f2a5e4ea295a2b71b70610b76, State: Running, Role: LEADER
I20260929 10:33:28.467381 21199 consensus_queue.cc:237] T 00000000000000000000000000000000 P b530997f2a5e4ea295a2b71b70610b76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b530997f2a5e4ea295a2b71b70610b76" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 46661 } }
I20260929 10:33:28.468034 21194 sys_catalog.cc:564] T 00000000000000000000000000000000 P b530997f2a5e4ea295a2b71b70610b76 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:33:28.469146 21201 sys_catalog.cc:455] T 00000000000000000000000000000000 P b530997f2a5e4ea295a2b71b70610b76 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b530997f2a5e4ea295a2b71b70610b76. Latest consensus state: current_term: 1 leader_uuid: "b530997f2a5e4ea295a2b71b70610b76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b530997f2a5e4ea295a2b71b70610b76" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 46661 } } }
I20260929 10:33:28.469280 21201 sys_catalog.cc:458] T 00000000000000000000000000000000 P b530997f2a5e4ea295a2b71b70610b76 [sys.catalog]: This master's current role is: LEADER
I20260929 10:33:28.469560 21200 sys_catalog.cc:455] T 00000000000000000000000000000000 P b530997f2a5e4ea295a2b71b70610b76 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b530997f2a5e4ea295a2b71b70610b76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b530997f2a5e4ea295a2b71b70610b76" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 46661 } } }
I20260929 10:33:28.469679 21200 sys_catalog.cc:458] T 00000000000000000000000000000000 P b530997f2a5e4ea295a2b71b70610b76 [sys.catalog]: This master's current role is: LEADER
I20260929 10:33:28.469765 21208 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:33:28.472584 21208 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:33:28.478714 21208 catalog_manager.cc:1347] Generated new cluster ID: 252caeea17934f7eb1cd38340cb1d6ca
I20260929 10:33:28.478792 21208 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:33:28.499682 21208 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:33:28.500972 21208 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:33:28.507764 21208 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b530997f2a5e4ea295a2b71b70610b76: Generated new TSK 0
I20260929 10:33:28.508574 21208 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 10:33:28.594622 21197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:28.523034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44000 (local address 127.15.82.126:46661)
0929 10:33:28.523172 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:28.523177 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:28.523368 (+ 191us) server_negotiation.cc:407] Connection header received
0929 10:33:28.523540 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:28.523542 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:28.523589 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:28.523670 (+ 81us) server_negotiation.cc:226] Negotiated authn=JWT
0929 10:33:28.523915 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:28.524440 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:28.525665 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:28.525878 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:28.525949 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:28.526274 (+ 325us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0929 10:33:28.594178 (+ 67904us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0929 10:33:28.594499 (+ 321us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44000: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":56}
W20260929 10:33:28.594724 21197 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44000: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260929 10:33:28.594883 21223 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:33:28.522964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:46661 (local address 127.0.0.1:44000)
0929 10:33:28.523276 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:28.523291 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:28.523405 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:28.523692 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:28.523695 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:28.523703 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0929 10:33:28.523815 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:28.523821 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:28.524547 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:28.524550 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:28.525544 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:28.525553 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:28.525993 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:28.525997 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:28.526091 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:28.526119 (+ 28us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0929 10:33:28.594459 (+ 68340us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0929 10:33:28.594697 (+ 238us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.15.82.126:46661: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":118,"threads_started":1}
I20260929 10:33:28.595512 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 21127
2026-09-29T10:33:28Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (362 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (122 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (123 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:33:28 dist-test-slave-vg89 krb5kdc[21235](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:33:28 dist-test-slave-vg89 krb5kdc[21235](info): set up 2 sockets
Sep 29 10:33:28 dist-test-slave-vg89 krb5kdc[21235](info): commencing operation
krb5kdc: starting...
W20260929 10:33:29.882941 15689 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Sep 29 10:33:29 dist-test-slave-vg89 krb5kdc[21235](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678009, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:33:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:33:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.82.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab.
Entry for principal kudu/127.15.82.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab.
I20260929 10:33:30.005971 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:42757
--webserver_interface=127.15.82.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab
--principal=kudu/127.15.82.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:44215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.82.126:42757
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:33:30.118934 21248 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:33:30.119295 21248 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:33:30.119378 21248 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:33:30.123553 21248 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:33:30.123642 21248 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:33:30.123682 21248 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:33:30.123745 21248 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:33:30.123786 21248 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:33:30.129168 21248 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:44215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.82.126:42757
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab
--principal=kudu/127.15.82.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:42757
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.82.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21248
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:33:30.130345 21248 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:33:30.131515 21248 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:33:30.140388 21253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:30.140388 21256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:30.140425 21254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:30.140805 21248 server_base.cc:1034] running on GCE node
I20260929 10:33:30.141356 21248 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:33:30.142560 21248 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:33:30.143721 21248 hybrid_clock.cc:648] HybridClock initialized: now 1790678010143704 us; error 33 us; skew 500 ppm
Sep 29 10:33:30 dist-test-slave-vg89 krb5kdc[21235](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678010, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:33:30.146705 21248 init.cc:377] Logged in from keytab as kudu/127.15.82.126@KRBTEST.COM (short username kudu)
I20260929 10:33:30.147590 21248 webserver.cc:460] Webserver started at http://127.15.82.126:38629/ using document root <none> and password file <none>
I20260929 10:33:30.148172 21248 fs_manager.cc:359] Metadata directory not provided
I20260929 10:33:30.148262 21248 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:33:30.148496 21248 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:33:30.150628 21248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/minicluster-data/master-0/data/instance:
uuid: "36b278a12ab340ffbc72e363f2a81ab3"
format_stamp: "Formatted at 2026-09-29 10:33:30 on dist-test-slave-vg89"
server_key: "9a0f0e79987f63a6776af5703e88017d"
server_key_iv: "35e15ad66b4bb5b85e197c200d509af8"
server_key_version: "encryptionkey@0"
I20260929 10:33:30.151227 21248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/minicluster-data/master-0/wal/instance:
uuid: "36b278a12ab340ffbc72e363f2a81ab3"
format_stamp: "Formatted at 2026-09-29 10:33:30 on dist-test-slave-vg89"
server_key: "9a0f0e79987f63a6776af5703e88017d"
server_key_iv: "35e15ad66b4bb5b85e197c200d509af8"
server_key_version: "encryptionkey@0"
I20260929 10:33:30.155148 21248 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 10:33:30.157547 21263 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:30.158931 21248 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 10:33:30.159070 21248 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/minicluster-data/master-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/minicluster-data/master-0/wal
uuid: "36b278a12ab340ffbc72e363f2a81ab3"
format_stamp: "Formatted at 2026-09-29 10:33:30 on dist-test-slave-vg89"
server_key: "9a0f0e79987f63a6776af5703e88017d"
server_key_iv: "35e15ad66b4bb5b85e197c200d509af8"
server_key_version: "encryptionkey@0"
I20260929 10:33:30.159194 21248 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:33:30.185045 21248 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:33:30.188735 21248 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:33:30.188963 21248 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:33:30.196484 21248 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.126:42757
I20260929 10:33:30.196501 21315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.126:42757 every 8 connection(s)
I20260929 10:33:30.197476 21248 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
I20260929 10:33:30.200388 21316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:33:30.203102 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 21248
I20260929 10:33:30.203238 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/minicluster-data/master-0/wal/instance
I20260929 10:33:30.203544 15689 external_mini_cluster.cc:1428] Setting key b0252453b255498c5d40df5a14a22b57
I20260929 10:33:30.206905 21316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36b278a12ab340ffbc72e363f2a81ab3: Bootstrap starting.
I20260929 10:33:30.209215 21316 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 36b278a12ab340ffbc72e363f2a81ab3: Neither blocks nor log segments found. Creating new log.
I20260929 10:33:30.209990 21316 log.cc:824] T 00000000000000000000000000000000 P 36b278a12ab340ffbc72e363f2a81ab3: Log is configured to *not* fsync() on all Append() calls
I20260929 10:33:30.212045 21316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36b278a12ab340ffbc72e363f2a81ab3: No bootstrap required, opened a new log
Sep 29 10:33:30 dist-test-slave-vg89 krb5kdc[21235](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678009, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:33:30.215929 21316 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36b278a12ab340ffbc72e363f2a81ab3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36b278a12ab340ffbc72e363f2a81ab3" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 42757 } }
I20260929 10:33:30.216248 21316 raft_consensus.cc:374] T 00000000000000000000000000000000 P 36b278a12ab340ffbc72e363f2a81ab3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:33:30.216333 21316 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36b278a12ab340ffbc72e363f2a81ab3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36b278a12ab340ffbc72e363f2a81ab3, State: Initialized, Role: FOLLOWER
I20260929 10:33:30.216856 21316 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36b278a12ab340ffbc72e363f2a81ab3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36b278a12ab340ffbc72e363f2a81ab3" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 42757 } }
I20260929 10:33:30.216991 21316 raft_consensus.cc:388] T 00000000000000000000000000000000 P 36b278a12ab340ffbc72e363f2a81ab3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:33:30.217092 21316 raft_consensus.cc:482] T 00000000000000000000000000000000 P 36b278a12ab340ffbc72e363f2a81ab3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:33:30.217222 21316 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36b278a12ab340ffbc72e363f2a81ab3 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:33:30.218382 21316 raft_consensus.cc:504] T 00000000000000000000000000000000 P 36b278a12ab340ffbc72e363f2a81ab3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36b278a12ab340ffbc72e363f2a81ab3" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 42757 } }
I20260929 10:33:30.218812 21316 leader_election.cc:302] T 00000000000000000000000000000000 P 36b278a12ab340ffbc72e363f2a81ab3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 36b278a12ab340ffbc72e363f2a81ab3; no voters:
I20260929 10:33:30.218946 21319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:30.205078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57564 (local address 127.15.82.126:42757)
0929 10:33:30.205527 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:30.205542 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:30.205579 (+ 37us) server_negotiation.cc:407] Connection header received
0929 10:33:30.206249 (+ 670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:30.206266 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:30.206685 (+ 419us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:30.207027 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:30.207862 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:30.208872 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:30.209807 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:30.210339 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:30.210581 (+ 242us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:30.213367 (+ 2786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:30.213373 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:30.213387 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:30.213416 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:30.215609 (+ 2193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:30.216136 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:30.216140 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:30.216146 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:30.216340 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:30.216678 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:30.216681 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:30.216683 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:30.217191 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:30.217395 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:30.217691 (+ 296us) server_negotiation.cc:299] Negotiation successful
0929 10:33:30.218003 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":126,"threads_started":1}
I20260929 10:33:30.219189 21316 leader_election.cc:288] T 00000000000000000000000000000000 P 36b278a12ab340ffbc72e363f2a81ab3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:33:30.219378 21321 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 36b278a12ab340ffbc72e363f2a81ab3 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:33:30.219592 21321 raft_consensus.cc:686] T 00000000000000000000000000000000 P 36b278a12ab340ffbc72e363f2a81ab3 [term 1 LEADER]: Becoming Leader. State: Replica: 36b278a12ab340ffbc72e363f2a81ab3, State: Running, Role: LEADER
I20260929 10:33:30.219964 21321 consensus_queue.cc:237] T 00000000000000000000000000000000 P 36b278a12ab340ffbc72e363f2a81ab3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36b278a12ab340ffbc72e363f2a81ab3" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 42757 } }
I20260929 10:33:30.221679 21322 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36b278a12ab340ffbc72e363f2a81ab3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "36b278a12ab340ffbc72e363f2a81ab3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36b278a12ab340ffbc72e363f2a81ab3" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 42757 } } }
I20260929 10:33:30.221722 21323 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36b278a12ab340ffbc72e363f2a81ab3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36b278a12ab340ffbc72e363f2a81ab3. Latest consensus state: current_term: 1 leader_uuid: "36b278a12ab340ffbc72e363f2a81ab3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36b278a12ab340ffbc72e363f2a81ab3" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 42757 } } }
I20260929 10:33:30.221804 21322 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36b278a12ab340ffbc72e363f2a81ab3 [sys.catalog]: This master's current role is: LEADER
I20260929 10:33:30.221827 21323 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36b278a12ab340ffbc72e363f2a81ab3 [sys.catalog]: This master's current role is: LEADER
I20260929 10:33:30.221968 21316 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36b278a12ab340ffbc72e363f2a81ab3 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:33:30.222144 21325 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:33:30.225399 21325 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:33:30.238817 21325 catalog_manager.cc:1347] Generated new cluster ID: dbe4b09477404bfa97cfd77156ea1786
I20260929 10:33:30.238967 21325 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:33:30.247066 21325 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:33:30.248342 21325 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:33:30.254447 21325 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 36b278a12ab340ffbc72e363f2a81ab3: Generated new TSK 0
I20260929 10:33:30.255264 21325 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.82.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab.
Entry for principal kudu/127.15.82.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab.
I20260929 10:33:30.289767 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.65:0
--local_ip_for_outbound_sockets=127.15.82.65
--webserver_interface=127.15.82.65
--webserver_port=0
--tserver_master_addrs=127.15.82.126:42757
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab
--principal=kudu/127.15.82.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:44215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:33:30.404474 21342 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:33:30.404757 21342 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:33:30.404803 21342 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:33:30.408954 21342 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:33:30.409018 21342 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:33:30.409085 21342 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.65
I20260929 10:33:30.414213 21342 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:44215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab
--principal=kudu/127.15.82.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.82.65
--webserver_port=0
--tserver_master_addrs=127.15.82.126:42757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21342
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.65
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:33:30.415330 21342 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:33:30.416378 21342 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:33:30.426033 21347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:30.426153 21350 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:30.426072 21348 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:30.426327 21342 server_base.cc:1034] running on GCE node
I20260929 10:33:30.426760 21342 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:33:30.427398 21342 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:33:30.428557 21342 hybrid_clock.cc:648] HybridClock initialized: now 1790678010428543 us; error 29 us; skew 500 ppm
Sep 29 10:33:30 dist-test-slave-vg89 krb5kdc[21235](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678010, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:33:30.431417 21342 init.cc:377] Logged in from keytab as kudu/127.15.82.65@KRBTEST.COM (short username kudu)
I20260929 10:33:30.432298 21342 webserver.cc:460] Webserver started at http://127.15.82.65:40867/ using document root <none> and password file <none>
I20260929 10:33:30.432806 21342 fs_manager.cc:359] Metadata directory not provided
I20260929 10:33:30.432864 21342 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:33:30.433022 21342 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:33:30.435058 21342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/minicluster-data/ts-0/data/instance:
uuid: "78b9d67def9f4cda9bdf2001f278f60f"
format_stamp: "Formatted at 2026-09-29 10:33:30 on dist-test-slave-vg89"
server_key: "8765637f9651d41e33baaf85d28ea038"
server_key_iv: "094fdc61a39bff8bd3b993452a414f9b"
server_key_version: "encryptionkey@0"
I20260929 10:33:30.435572 21342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/minicluster-data/ts-0/wal/instance:
uuid: "78b9d67def9f4cda9bdf2001f278f60f"
format_stamp: "Formatted at 2026-09-29 10:33:30 on dist-test-slave-vg89"
server_key: "8765637f9651d41e33baaf85d28ea038"
server_key_iv: "094fdc61a39bff8bd3b993452a414f9b"
server_key_version: "encryptionkey@0"
I20260929 10:33:30.439216 21342 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260929 10:33:30.441707 21357 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:30.442958 21342 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 10:33:30.443094 21342 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/minicluster-data/ts-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/minicluster-data/ts-0/wal
uuid: "78b9d67def9f4cda9bdf2001f278f60f"
format_stamp: "Formatted at 2026-09-29 10:33:30 on dist-test-slave-vg89"
server_key: "8765637f9651d41e33baaf85d28ea038"
server_key_iv: "094fdc61a39bff8bd3b993452a414f9b"
server_key_version: "encryptionkey@0"
I20260929 10:33:30.443233 21342 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:33:30.462613 21342 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:33:30.465975 21342 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:33:30.466207 21342 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:33:30.466841 21342 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:33:30.467703 21342 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:33:30.467775 21342 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:30.467849 21342 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:33:30.467891 21342 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:30.476557 21342 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.65:45151
I20260929 10:33:30.476617 21470 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.65:45151 every 8 connection(s)
I20260929 10:33:30.477543 21342 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
I20260929 10:33:30.486742 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 21342
I20260929 10:33:30.486878 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/minicluster-data/ts-0/wal/instance
Sep 29 10:33:30 dist-test-slave-vg89 krb5kdc[21235](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678010, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.65@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:33:30.487205 15689 external_mini_cluster.cc:1428] Setting key ad4f4955bc7bfe34199085aff8a48a12
I20260929 10:33:30.491755 21319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:30.479422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.65:35139 (local address 127.15.82.126:42757)
0929 10:33:30.479595 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:30.479600 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:30.480447 (+ 847us) server_negotiation.cc:407] Connection header received
0929 10:33:30.481758 (+ 1311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:30.481761 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:30.481814 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:30.481903 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:30.483190 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:30.483756 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:30.484865 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:30.485114 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:30.485210 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:30.487915 (+ 2705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:30.487918 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:30.487920 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:30.487943 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:30.489445 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:30.490080 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:30.490082 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:30.490083 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:30.490157 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:30.490615 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:30.490620 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:30.490622 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:30.490811 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:30.490922 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:30.491447 (+ 525us) server_negotiation.cc:299] Negotiation successful
0929 10:33:30.491574 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260929 10:33:30.492584 21473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:30.479830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:42757 (local address 127.15.82.65:35139)
0929 10:33:30.480289 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:30.480332 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:30.481540 (+ 1208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:30.482064 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:30.482070 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:30.482496 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:30.483025 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:30.483039 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:30.483872 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:30.483874 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:30.484744 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:30.484752 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:30.485260 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:30.485262 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:30.485396 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:30.485943 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:30.485959 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:30.487712 (+ 1753us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:30.489564 (+ 1852us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:30.489574 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:30.489593 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:30.489972 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:30.490257 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:30.490263 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:30.490268 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:30.490497 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:30.490922 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:30.490931 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:30.491198 (+ 267us) client_negotiation.cc:769] Sending connection context
0929 10:33:30.491391 (+ 193us) client_negotiation.cc:240] Negotiation successful
0929 10:33:30.491731 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":113,"threads_started":1}
I20260929 10:33:30.493772 21471 heartbeater.cc:347] Connected to a master server at 127.15.82.126:42757
I20260929 10:33:30.494021 21471 heartbeater.cc:464] Registering TS with master...
I20260929 10:33:30.494585 21471 heartbeater.cc:511] Master 127.15.82.126:42757 requested a full tablet report, sending...
I20260929 10:33:30.496104 21280 ts_manager.cc:194] Registered new tserver with Master: 78b9d67def9f4cda9bdf2001f278f60f (127.15.82.65:45151)
I20260929 10:33:30.497232 21280 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.82.65@KRBTEST.COM'} at 127.15.82.65:35139
WARNING: no policy specified for kudu/127.15.82.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab.
Entry for principal kudu/127.15.82.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab.
I20260929 10:33:30.515370 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.66:0
--local_ip_for_outbound_sockets=127.15.82.66
--webserver_interface=127.15.82.66
--webserver_port=0
--tserver_master_addrs=127.15.82.126:42757
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab
--principal=kudu/127.15.82.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:44215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:33:30.625442 21476 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:33:30.625696 21476 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:33:30.625787 21476 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:33:30.629765 21476 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:33:30.629819 21476 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:33:30.629938 21476 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.66
I20260929 10:33:30.635093 21476 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:44215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab
--principal=kudu/127.15.82.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.82.66
--webserver_port=0
--tserver_master_addrs=127.15.82.126:42757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21476
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.66
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:33:30.636127 21476 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:33:30.637202 21476 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:33:30.646693 21482 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:30.646795 21481 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:30.646704 21484 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:30.646780 21476 server_base.cc:1034] running on GCE node
I20260929 10:33:30.647390 21476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:33:30.647993 21476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:33:30.649163 21476 hybrid_clock.cc:648] HybridClock initialized: now 1790678010649143 us; error 44 us; skew 500 ppm
Sep 29 10:33:30 dist-test-slave-vg89 krb5kdc[21235](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678010, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:33:30.652009 21476 init.cc:377] Logged in from keytab as kudu/127.15.82.66@KRBTEST.COM (short username kudu)
I20260929 10:33:30.652884 21476 webserver.cc:460] Webserver started at http://127.15.82.66:45441/ using document root <none> and password file <none>
I20260929 10:33:30.653472 21476 fs_manager.cc:359] Metadata directory not provided
I20260929 10:33:30.653568 21476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:33:30.653803 21476 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:33:30.655939 21476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/minicluster-data/ts-1/data/instance:
uuid: "f986530b136d4963a4165ffd34e235cf"
format_stamp: "Formatted at 2026-09-29 10:33:30 on dist-test-slave-vg89"
server_key: "07046a81b6312dc226e1426f13756742"
server_key_iv: "006324bb71a338b8bf3b7dd00fa98cd0"
server_key_version: "encryptionkey@0"
I20260929 10:33:30.656528 21476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/minicluster-data/ts-1/wal/instance:
uuid: "f986530b136d4963a4165ffd34e235cf"
format_stamp: "Formatted at 2026-09-29 10:33:30 on dist-test-slave-vg89"
server_key: "07046a81b6312dc226e1426f13756742"
server_key_iv: "006324bb71a338b8bf3b7dd00fa98cd0"
server_key_version: "encryptionkey@0"
I20260929 10:33:30.660367 21476 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 10:33:30.662874 21491 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:30.664141 21476 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 10:33:30.664295 21476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/minicluster-data/ts-1/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/minicluster-data/ts-1/wal
uuid: "f986530b136d4963a4165ffd34e235cf"
format_stamp: "Formatted at 2026-09-29 10:33:30 on dist-test-slave-vg89"
server_key: "07046a81b6312dc226e1426f13756742"
server_key_iv: "006324bb71a338b8bf3b7dd00fa98cd0"
server_key_version: "encryptionkey@0"
I20260929 10:33:30.664412 21476 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:33:30.678953 21476 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:33:30.682106 21476 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:33:30.682329 21476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:33:30.682998 21476 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:33:30.683840 21476 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:33:30.683887 21476 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:30.683926 21476 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:33:30.683941 21476 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:30.692188 21476 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.66:34699
I20260929 10:33:30.692277 21604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.66:34699 every 8 connection(s)
I20260929 10:33:30.693127 21476 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
I20260929 10:33:30.701718 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 21476
I20260929 10:33:30.701869 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/minicluster-data/ts-1/wal/instance
I20260929 10:33:30.702163 15689 external_mini_cluster.cc:1428] Setting key 2d2e40ab9c1b07e80ccb6845395f4d68
Sep 29 10:33:30 dist-test-slave-vg89 krb5kdc[21235](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678010, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.66@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:33:30.708037 21319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:30.695028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:51729 (local address 127.15.82.126:42757)
0929 10:33:30.695179 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:30.695184 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:30.696122 (+ 938us) server_negotiation.cc:407] Connection header received
0929 10:33:30.697470 (+ 1348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:30.697472 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:30.697521 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:30.697599 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:30.698883 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:30.699420 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:30.700603 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:30.700852 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:30.700954 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:30.703901 (+ 2947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:30.703905 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:30.703908 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:30.703935 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:30.705596 (+ 1661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:30.706264 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:30.706267 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:30.706268 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:30.706342 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:30.706761 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:30.706763 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:30.706764 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:30.706937 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:30.707013 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:30.707641 (+ 628us) server_negotiation.cc:299] Negotiation successful
0929 10:33:30.707787 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260929 10:33:30.708734 21607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:30.695462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:42757 (local address 127.15.82.66:51729)
0929 10:33:30.695913 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:30.695957 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:30.697265 (+ 1308us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:30.697778 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:30.697784 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:30.698151 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:30.698741 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:30.698762 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:30.699533 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:30.699535 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:30.700447 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:30.700456 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:30.701002 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:30.701004 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:30.701146 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:30.701940 (+ 794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:30.701968 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:30.703742 (+ 1774us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:30.705736 (+ 1994us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:30.705745 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:30.705764 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:30.706156 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:30.706463 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:30.706468 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:30.706473 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:30.706665 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:30.707046 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:30.707053 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:30.707374 (+ 321us) client_negotiation.cc:769] Sending connection context
0929 10:33:30.707533 (+ 159us) client_negotiation.cc:240] Negotiation successful
0929 10:33:30.707794 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":108,"threads_started":1}
I20260929 10:33:30.710007 21605 heartbeater.cc:347] Connected to a master server at 127.15.82.126:42757
I20260929 10:33:30.710278 21605 heartbeater.cc:464] Registering TS with master...
I20260929 10:33:30.710829 21605 heartbeater.cc:511] Master 127.15.82.126:42757 requested a full tablet report, sending...
I20260929 10:33:30.711957 21280 ts_manager.cc:194] Registered new tserver with Master: f986530b136d4963a4165ffd34e235cf (127.15.82.66:34699)
WARNING: no policy specified for kudu/127.15.82.67@KRBTEST.COM; defaulting to no policy
I20260929 10:33:30.712805 21280 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.82.66@KRBTEST.COM'} at 127.15.82.66:51729
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab.
Entry for principal kudu/127.15.82.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab.
I20260929 10:33:30.730633 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.67:0
--local_ip_for_outbound_sockets=127.15.82.67
--webserver_interface=127.15.82.67
--webserver_port=0
--tserver_master_addrs=127.15.82.126:42757
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab
--principal=kudu/127.15.82.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:44215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:33:30.841846 21610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:33:30.842118 21610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:33:30.842160 21610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:33:30.846561 21610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:33:30.846645 21610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:33:30.846724 21610 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.67
I20260929 10:33:30.852137 21610 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:44215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab
--principal=kudu/127.15.82.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.82.67
--webserver_port=0
--tserver_master_addrs=127.15.82.126:42757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21610
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.67
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:33:30.853169 21610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:33:30.854139 21610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:33:30.863883 21615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:30.863883 21616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:30.863940 21618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:30.864238 21610 server_base.cc:1034] running on GCE node
I20260929 10:33:30.864657 21610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:33:30.865269 21610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:33:30.866454 21610 hybrid_clock.cc:648] HybridClock initialized: now 1790678010866413 us; error 55 us; skew 500 ppm
Sep 29 10:33:30 dist-test-slave-vg89 krb5kdc[21235](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678010, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:33:30.869431 21610 init.cc:377] Logged in from keytab as kudu/127.15.82.67@KRBTEST.COM (short username kudu)
I20260929 10:33:30.870251 21610 webserver.cc:460] Webserver started at http://127.15.82.67:42983/ using document root <none> and password file <none>
I20260929 10:33:30.870847 21610 fs_manager.cc:359] Metadata directory not provided
I20260929 10:33:30.870908 21610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:33:30.871111 21610 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:33:30.873085 21610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/minicluster-data/ts-2/data/instance:
uuid: "b04eb1e0b1374795925ca9a7e1f9f81c"
format_stamp: "Formatted at 2026-09-29 10:33:30 on dist-test-slave-vg89"
server_key: "5f708d81d6e03f83bdb58f0967b80676"
server_key_iv: "f54f840d8a379de81359c610755074bf"
server_key_version: "encryptionkey@0"
I20260929 10:33:30.873631 21610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/minicluster-data/ts-2/wal/instance:
uuid: "b04eb1e0b1374795925ca9a7e1f9f81c"
format_stamp: "Formatted at 2026-09-29 10:33:30 on dist-test-slave-vg89"
server_key: "5f708d81d6e03f83bdb58f0967b80676"
server_key_iv: "f54f840d8a379de81359c610755074bf"
server_key_version: "encryptionkey@0"
I20260929 10:33:30.877206 21610 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260929 10:33:30.879668 21625 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:30.880842 21610 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260929 10:33:30.880967 21610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/minicluster-data/ts-2/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/minicluster-data/ts-2/wal
uuid: "b04eb1e0b1374795925ca9a7e1f9f81c"
format_stamp: "Formatted at 2026-09-29 10:33:30 on dist-test-slave-vg89"
server_key: "5f708d81d6e03f83bdb58f0967b80676"
server_key_iv: "f54f840d8a379de81359c610755074bf"
server_key_version: "encryptionkey@0"
I20260929 10:33:30.881069 21610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:33:30.899552 21610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:33:30.902292 21610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:33:30.902511 21610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:33:30.903085 21610 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:33:30.903906 21610 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:33:30.903980 21610 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:30.904045 21610 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:33:30.904093 21610 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:30.912947 21610 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.67:40823
I20260929 10:33:30.912966 21738 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.67:40823 every 8 connection(s)
I20260929 10:33:30.913822 21610 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
I20260929 10:33:30.917099 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 21610
I20260929 10:33:30.917193 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/minicluster-data/ts-2/wal/instance
I20260929 10:33:30.917481 15689 external_mini_cluster.cc:1428] Setting key 755aa7abfcca15a9979fa5234d922c5c
Sep 29 10:33:30 dist-test-slave-vg89 krb5kdc[21235](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678010, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.67@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:33:30.927536 21319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:30.915630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.67:34951 (local address 127.15.82.126:42757)
0929 10:33:30.915780 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:30.915784 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:30.916603 (+ 819us) server_negotiation.cc:407] Connection header received
0929 10:33:30.918120 (+ 1517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:30.918123 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:30.918178 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:30.918257 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:30.919445 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:30.920006 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:30.921139 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:30.921365 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:30.921477 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:30.924114 (+ 2637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:30.924117 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:30.924119 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:30.924142 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:30.925468 (+ 1326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:30.925985 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:30.925989 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:30.925991 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:30.926072 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:30.926437 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:30.926440 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:30.926442 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:30.926617 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:30.926736 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:30.927259 (+ 523us) server_negotiation.cc:299] Negotiation successful
0929 10:33:30.927378 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260929 10:33:30.928319 21741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:30.916011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:42757 (local address 127.15.82.67:34951)
0929 10:33:30.916453 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:30.916495 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:30.917894 (+ 1399us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:30.918472 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:30.918480 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:30.918827 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:30.919306 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:30.919319 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:30.920135 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:30.920138 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:30.921013 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:30.921021 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:30.921513 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:30.921517 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:30.921688 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:30.922323 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:30.922338 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:30.923954 (+ 1616us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:30.925585 (+ 1631us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:30.925591 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:30.925605 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:30.925881 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:30.926163 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:30.926176 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:30.926178 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:30.926310 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:30.926731 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:30.926738 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:30.927012 (+ 274us) client_negotiation.cc:769] Sending connection context
0929 10:33:30.927216 (+ 204us) client_negotiation.cc:240] Negotiation successful
0929 10:33:30.927529 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":114,"threads_started":1}
I20260929 10:33:30.929489 21739 heartbeater.cc:347] Connected to a master server at 127.15.82.126:42757
I20260929 10:33:30.929736 21739 heartbeater.cc:464] Registering TS with master...
I20260929 10:33:30.930322 21739 heartbeater.cc:511] Master 127.15.82.126:42757 requested a full tablet report, sending...
I20260929 10:33:30.931406 21280 ts_manager.cc:194] Registered new tserver with Master: b04eb1e0b1374795925ca9a7e1f9f81c (127.15.82.67:40823)
I20260929 10:33:30.931967 21280 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.82.67@KRBTEST.COM'} at 127.15.82.67:34951
I20260929 10:33:30.943102 15689 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/krb5kdc/test-admin.keytab.
Sep 29 10:33:30 dist-test-slave-vg89 krb5kdc[21235](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678010, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:33:30.959285 15689 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260929 10:33:30.961999 21746 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:33:30.960642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59086 (local address 127.15.82.65:45151)
0929 10:33:30.960933 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:30.960947 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:30.960971 (+ 24us) server_negotiation.cc:407] Connection header received
0929 10:33:30.961022 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:30.961027 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:30.961210 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:30.961327 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:30.961823 (+ 496us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59086: BlockingRecv error: recv got EOF from 127.0.0.1:59086 (error 108)
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":102,"threads_started":1}
W20260929 10:33:30.962075 15689 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/krb5kdc/krb5cc' of type FILE
Sep 29 10:33:30 dist-test-slave-vg89 krb5kdc[21235](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678010, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:33:30.963235 15689 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 29 10:33:30 dist-test-slave-vg89 krb5kdc[21235](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678010, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:33:30.964186 15689 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 29 10:33:30 dist-test-slave-vg89 krb5kdc[21235](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678010, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.82.65@KRBTEST.COM
I20260929 10:33:30.973980 21746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:30.965028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59100 (local address 127.15.82.65:45151)
0929 10:33:30.965152 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:30.965156 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:30.965270 (+ 114us) server_negotiation.cc:407] Connection header received
0929 10:33:30.965413 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:30.965416 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:30.965465 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:30.965551 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:30.966074 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:30.966930 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:30.967892 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:30.968283 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:30.968402 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:30.970707 (+ 2305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:30.970716 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:30.970732 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:30.970755 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:30.972308 (+ 1553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:30.972707 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:30.972713 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:30.972714 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:30.972800 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:30.973057 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:30.973060 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:30.973061 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:30.973313 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:30.973478 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:30.973723 (+ 245us) server_negotiation.cc:299] Negotiation successful
0929 10:33:30.973851 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
W20260929 10:33:30.977756 21746 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:33:30.976959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59108 (local address 127.15.82.65:45151)
0929 10:33:30.977097 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:30.977099 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:30.977111 (+ 12us) server_negotiation.cc:407] Connection header received
0929 10:33:30.977224 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:30.977226 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:30.977260 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:30.977320 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:30.977658 (+ 338us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59108: BlockingRecv error: recv got EOF from 127.0.0.1:59108 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260929 10:33:30.977941 15689 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/krb5kdc/krb5cc' of type FILE
Sep 29 10:33:30 dist-test-slave-vg89 krb5kdc[21235](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678010, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:33:30.978979 15689 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 29 10:33:30 dist-test-slave-vg89 krb5kdc[21235](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678010, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:33:30.979889 15689 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 29 10:33:30 dist-test-slave-vg89 krb5kdc[21235](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678010, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.82.65@KRBTEST.COM
I20260929 10:33:30.988951 21746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:30.980655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59116 (local address 127.15.82.65:45151)
0929 10:33:30.980782 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:30.980787 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:30.980913 (+ 126us) server_negotiation.cc:407] Connection header received
0929 10:33:30.980980 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:30.980982 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:30.981019 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:30.981121 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:30.981567 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:30.982120 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:30.982945 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:30.983364 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:30.983484 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:30.986033 (+ 2549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:30.986036 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:30.986038 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:30.986058 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:30.987360 (+ 1302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:30.987788 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:30.987796 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:30.987798 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:30.987867 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:30.988146 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:30.988150 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:30.988151 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:30.988311 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:30.988442 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:30.988686 (+ 244us) server_negotiation.cc:299] Negotiation successful
0929 10:33:30.988800 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
W20260929 10:33:30.991433 21746 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:33:30.990581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59130 (local address 127.15.82.65:45151)
0929 10:33:30.990698 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:30.990702 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:30.990845 (+ 143us) server_negotiation.cc:407] Connection header received
0929 10:33:30.990915 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:30.990917 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:30.990951 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:30.991021 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:30.991335 (+ 314us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59130: BlockingRecv error: recv got EOF from 127.0.0.1:59130 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260929 10:33:30.991559 15689 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790677981592896-15689-0/krb5kdc/krb5cc' of type FILE
Sep 29 10:33:30 dist-test-slave-vg89 krb5kdc[21235](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678010, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:33:30.992511 15689 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 29 10:33:30 dist-test-slave-vg89 krb5kdc[21235](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678010, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:33:30.993363 15689 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 29 10:33:30 dist-test-slave-vg89 krb5kdc[21235](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678010, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.82.65@KRBTEST.COM
I20260929 10:33:31.001542 21746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:30.994132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59144 (local address 127.15.82.65:45151)
0929 10:33:30.994253 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:30.994257 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:30.994378 (+ 121us) server_negotiation.cc:407] Connection header received
0929 10:33:30.994546 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:30.994551 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:30.994594 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:30.994677 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:30.995159 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:30.995647 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:30.996432 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:30.996648 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:30.996753 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:30.998883 (+ 2130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:30.998886 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:30.998888 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:30.998909 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:31.000196 (+ 1287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:31.000562 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:31.000565 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:31.000566 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:31.000636 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:31.000854 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:31.000856 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:31.000857 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:31.001008 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:31.001139 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:31.001337 (+ 198us) server_negotiation.cc:299] Negotiation successful
0929 10:33:31.001426 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260929 10:33:31.002564 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 21342
I20260929 10:33:31.008189 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 21476
I20260929 10:33:31.013689 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 21610
I20260929 10:33:31.019057 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 21248
2026-09-29T10:33:31Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2179 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:33:31 dist-test-slave-vg89 krb5kdc[21759](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:33:31 dist-test-slave-vg89 krb5kdc[21759](info): set up 2 sockets
Sep 29 10:33:31 dist-test-slave-vg89 krb5kdc[21759](info): commencing operation
krb5kdc: starting...
W20260929 10:33:32.059137 15689 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Sep 29 10:33:32 dist-test-slave-vg89 krb5kdc[21759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678012, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:33:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:33:32Z Disabled control of system clock
WARNING: no policy specified for oryx/127.15.82.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.82.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.82.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/krb5kdc/oryx_127.15.82.126.keytab.
Entry for principal oryx/127.15.82.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/krb5kdc/oryx_127.15.82.126.keytab.
I20260929 10:33:32.181186 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:42245
--webserver_interface=127.15.82.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/krb5kdc/oryx_127.15.82.126.keytab
--principal=oryx/127.15.82.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:40517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.82.126:42245
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:33:32.292264 21772 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:33:32.292527 21772 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:33:32.292567 21772 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:33:32.296785 21772 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:33:32.296838 21772 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:33:32.296859 21772 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:33:32.296875 21772 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:33:32.296927 21772 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:33:32.302234 21772 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:40517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.82.126:42245
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/krb5kdc/oryx_127.15.82.126.keytab
--principal=oryx/127.15.82.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:42245
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.82.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21772
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:33:32.303335 21772 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:33:32.304373 21772 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:33:32.313066 21778 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:32.313066 21780 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:32.313235 21772 server_base.cc:1034] running on GCE node
W20260929 10:33:32.313247 21777 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:32.313853 21772 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:33:32.314996 21772 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:33:32.316157 21772 hybrid_clock.cc:648] HybridClock initialized: now 1790678012316142 us; error 27 us; skew 500 ppm
Sep 29 10:33:32 dist-test-slave-vg89 krb5kdc[21759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678012, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.82.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:33:32.318883 21772 init.cc:377] Logged in from keytab as oryx/127.15.82.126@KRBTEST.COM (short username oryx)
I20260929 10:33:32.319811 21772 webserver.cc:460] Webserver started at http://127.15.82.126:40085/ using document root <none> and password file <none>
I20260929 10:33:32.320391 21772 fs_manager.cc:359] Metadata directory not provided
I20260929 10:33:32.320452 21772 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:33:32.320672 21772 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:33:32.322793 21772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/master-0/data/instance:
uuid: "2b919a4b6dcc4702b515509aa1dd2164"
format_stamp: "Formatted at 2026-09-29 10:33:32 on dist-test-slave-vg89"
server_key: "778783accaa30fb86d5185f1301ec42d"
server_key_iv: "ea57e9709ac7e0b6fcf6f51a159ce898"
server_key_version: "encryptionkey@0"
I20260929 10:33:32.323357 21772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/master-0/wal/instance:
uuid: "2b919a4b6dcc4702b515509aa1dd2164"
format_stamp: "Formatted at 2026-09-29 10:33:32 on dist-test-slave-vg89"
server_key: "778783accaa30fb86d5185f1301ec42d"
server_key_iv: "ea57e9709ac7e0b6fcf6f51a159ce898"
server_key_version: "encryptionkey@0"
I20260929 10:33:32.327034 21772 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260929 10:33:32.329365 21787 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:32.330622 21772 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260929 10:33:32.330756 21772 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/master-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/master-0/wal
uuid: "2b919a4b6dcc4702b515509aa1dd2164"
format_stamp: "Formatted at 2026-09-29 10:33:32 on dist-test-slave-vg89"
server_key: "778783accaa30fb86d5185f1301ec42d"
server_key_iv: "ea57e9709ac7e0b6fcf6f51a159ce898"
server_key_version: "encryptionkey@0"
I20260929 10:33:32.330875 21772 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:33:32.351239 21772 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:33:32.354933 21772 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:33:32.355207 21772 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:33:32.362607 21772 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.126:42245
I20260929 10:33:32.362905 21839 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.126:42245 every 8 connection(s)
I20260929 10:33:32.363620 21772 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
I20260929 10:33:32.366466 21840 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:33:32.367780 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 21772
I20260929 10:33:32.367913 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/master-0/wal/instance
I20260929 10:33:32.368237 15689 external_mini_cluster.cc:1428] Setting key 5dada986e0892592477bafdb1a34ee07
I20260929 10:33:32.373250 21840 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b919a4b6dcc4702b515509aa1dd2164: Bootstrap starting.
I20260929 10:33:32.376025 21840 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2b919a4b6dcc4702b515509aa1dd2164: Neither blocks nor log segments found. Creating new log.
I20260929 10:33:32.376967 21840 log.cc:824] T 00000000000000000000000000000000 P 2b919a4b6dcc4702b515509aa1dd2164: Log is configured to *not* fsync() on all Append() calls
Sep 29 10:33:32 dist-test-slave-vg89 krb5kdc[21759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678012, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM
I20260929 10:33:32.379019 21840 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b919a4b6dcc4702b515509aa1dd2164: No bootstrap required, opened a new log
I20260929 10:33:32.382727 21843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:32.369657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44826 (local address 127.15.82.126:42245)
0929 10:33:32.370058 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:32.370090 (+ 32us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:32.370125 (+ 35us) server_negotiation.cc:407] Connection header received
0929 10:33:32.370820 (+ 695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:32.370838 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:32.371182 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:32.371458 (+ 276us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:32.372369 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:32.373385 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:32.374514 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:32.374976 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:32.375093 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:32.377962 (+ 2869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:32.377966 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:32.377979 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:32.378003 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:32.379790 (+ 1787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:32.380266 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:32.380270 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:32.380274 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:32.380379 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:32.380748 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:32.380751 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:32.380752 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:32.381127 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:32.381277 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:32.381600 (+ 323us) server_negotiation.cc:299] Negotiation successful
0929 10:33:32.381926 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":146,"threads_started":1}
I20260929 10:33:32.383260 21840 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2b919a4b6dcc4702b515509aa1dd2164 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b919a4b6dcc4702b515509aa1dd2164" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 42245 } }
I20260929 10:33:32.383526 21840 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2b919a4b6dcc4702b515509aa1dd2164 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:33:32.383589 21840 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2b919a4b6dcc4702b515509aa1dd2164 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b919a4b6dcc4702b515509aa1dd2164, State: Initialized, Role: FOLLOWER
I20260929 10:33:32.384030 21840 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2b919a4b6dcc4702b515509aa1dd2164 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b919a4b6dcc4702b515509aa1dd2164" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 42245 } }
I20260929 10:33:32.384189 21840 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2b919a4b6dcc4702b515509aa1dd2164 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:33:32.384271 21840 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2b919a4b6dcc4702b515509aa1dd2164 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:33:32.384397 21840 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b919a4b6dcc4702b515509aa1dd2164 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:33:32.385504 21840 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2b919a4b6dcc4702b515509aa1dd2164 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b919a4b6dcc4702b515509aa1dd2164" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 42245 } }
I20260929 10:33:32.385849 21840 leader_election.cc:302] T 00000000000000000000000000000000 P 2b919a4b6dcc4702b515509aa1dd2164 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2b919a4b6dcc4702b515509aa1dd2164; no voters:
I20260929 10:33:32.386160 21840 leader_election.cc:288] T 00000000000000000000000000000000 P 2b919a4b6dcc4702b515509aa1dd2164 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:33:32.386298 21845 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2b919a4b6dcc4702b515509aa1dd2164 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:33:32.386590 21845 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2b919a4b6dcc4702b515509aa1dd2164 [term 1 LEADER]: Becoming Leader. State: Replica: 2b919a4b6dcc4702b515509aa1dd2164, State: Running, Role: LEADER
I20260929 10:33:32.386929 21845 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2b919a4b6dcc4702b515509aa1dd2164 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b919a4b6dcc4702b515509aa1dd2164" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 42245 } }
I20260929 10:33:32.387391 21840 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2b919a4b6dcc4702b515509aa1dd2164 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:33:32.388696 21846 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b919a4b6dcc4702b515509aa1dd2164 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2b919a4b6dcc4702b515509aa1dd2164" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b919a4b6dcc4702b515509aa1dd2164" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 42245 } } }
I20260929 10:33:32.388710 21847 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b919a4b6dcc4702b515509aa1dd2164 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b919a4b6dcc4702b515509aa1dd2164. Latest consensus state: current_term: 1 leader_uuid: "2b919a4b6dcc4702b515509aa1dd2164" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b919a4b6dcc4702b515509aa1dd2164" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 42245 } } }
I20260929 10:33:32.388840 21847 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b919a4b6dcc4702b515509aa1dd2164 [sys.catalog]: This master's current role is: LEADER
I20260929 10:33:32.388840 21846 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b919a4b6dcc4702b515509aa1dd2164 [sys.catalog]: This master's current role is: LEADER
W20260929 10:33:32.392351 21860 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2b919a4b6dcc4702b515509aa1dd2164: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 10:33:32.392417 21860 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 10:33:32.392496 21856 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:33:32.393183 21856 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:33:32.399147 21856 catalog_manager.cc:1347] Generated new cluster ID: 1f355f33ab054e5c8cc0fe255d71c834
I20260929 10:33:32.399215 21856 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:33:32.417197 21856 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:33:32.418321 21856 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:33:32.424077 21856 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2b919a4b6dcc4702b515509aa1dd2164: Generated new TSK 0
I20260929 10:33:32.424950 21856 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.15.82.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.82.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.82.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/krb5kdc/oryx_127.15.82.65.keytab.
Entry for principal oryx/127.15.82.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/krb5kdc/oryx_127.15.82.65.keytab.
I20260929 10:33:32.453524 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.65:0
--local_ip_for_outbound_sockets=127.15.82.65
--webserver_interface=127.15.82.65
--webserver_port=0
--tserver_master_addrs=127.15.82.126:42245
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/krb5kdc/oryx_127.15.82.65.keytab
--principal=oryx/127.15.82.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:40517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:33:32.564733 21866 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:33:32.564992 21866 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:33:32.565037 21866 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:33:32.569186 21866 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:33:32.569243 21866 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:33:32.569304 21866 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.65
I20260929 10:33:32.574708 21866 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:40517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/krb5kdc/oryx_127.15.82.65.keytab
--principal=oryx/127.15.82.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.82.65
--webserver_port=0
--tserver_master_addrs=127.15.82.126:42245
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21866
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.65
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:33:32.575867 21866 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:33:32.576941 21866 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:33:32.586788 21871 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:32.586795 21872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:32.586973 21866 server_base.cc:1034] running on GCE node
W20260929 10:33:32.587141 21874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:32.587576 21866 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:33:32.588285 21866 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:33:32.589530 21866 hybrid_clock.cc:648] HybridClock initialized: now 1790678012589500 us; error 33 us; skew 500 ppm
Sep 29 10:33:32 dist-test-slave-vg89 krb5kdc[21759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678012, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.82.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:33:32.592271 21866 init.cc:377] Logged in from keytab as oryx/127.15.82.65@KRBTEST.COM (short username oryx)
I20260929 10:33:32.593061 21866 webserver.cc:460] Webserver started at http://127.15.82.65:35231/ using document root <none> and password file <none>
I20260929 10:33:32.593573 21866 fs_manager.cc:359] Metadata directory not provided
I20260929 10:33:32.593631 21866 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:33:32.593799 21866 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:33:32.595822 21866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-0/data/instance:
uuid: "068171b1b6514121936f4634de5a8064"
format_stamp: "Formatted at 2026-09-29 10:33:32 on dist-test-slave-vg89"
server_key: "8249a5889aa23be947694a761cd99c0a"
server_key_iv: "2ecebf5ab4709fb59969bfb0b84efb15"
server_key_version: "encryptionkey@0"
I20260929 10:33:32.596323 21866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-0/wal/instance:
uuid: "068171b1b6514121936f4634de5a8064"
format_stamp: "Formatted at 2026-09-29 10:33:32 on dist-test-slave-vg89"
server_key: "8249a5889aa23be947694a761cd99c0a"
server_key_iv: "2ecebf5ab4709fb59969bfb0b84efb15"
server_key_version: "encryptionkey@0"
I20260929 10:33:32.599974 21866 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260929 10:33:32.602299 21881 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:32.603515 21866 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 10:33:32.603610 21866 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-0/wal
uuid: "068171b1b6514121936f4634de5a8064"
format_stamp: "Formatted at 2026-09-29 10:33:32 on dist-test-slave-vg89"
server_key: "8249a5889aa23be947694a761cd99c0a"
server_key_iv: "2ecebf5ab4709fb59969bfb0b84efb15"
server_key_version: "encryptionkey@0"
I20260929 10:33:32.603703 21866 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:33:32.627162 21866 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:33:32.630666 21866 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:33:32.630891 21866 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:33:32.631523 21866 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:33:32.632371 21866 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:33:32.632419 21866 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:32.632489 21866 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:33:32.632519 21866 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:32.641826 21866 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.65:45833
I20260929 10:33:32.641861 21994 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.65:45833 every 8 connection(s)
I20260929 10:33:32.642823 21866 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
I20260929 10:33:32.650164 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 21866
I20260929 10:33:32.650324 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-0/wal/instance
I20260929 10:33:32.650650 15689 external_mini_cluster.cc:1428] Setting key a8638fa2b08811c36d43605c36f3b620
Sep 29 10:33:32 dist-test-slave-vg89 krb5kdc[21759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678012, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.82.65@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM
I20260929 10:33:32.657805 21843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:32.644754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.65:53995 (local address 127.15.82.126:42245)
0929 10:33:32.644908 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:32.644912 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:32.645790 (+ 878us) server_negotiation.cc:407] Connection header received
0929 10:33:32.647246 (+ 1456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:32.647249 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:32.647308 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:32.647389 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:32.648573 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:32.649122 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:32.650275 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:32.650630 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:32.650748 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:32.653839 (+ 3091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:32.653842 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:32.653844 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:32.653869 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:32.655567 (+ 1698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:32.656059 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:32.656066 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:32.656071 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:32.656182 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:32.656501 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:32.656507 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:32.656512 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:32.656777 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:32.656894 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:32.657435 (+ 541us) server_negotiation.cc:299] Negotiation successful
0929 10:33:32.657580 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260929 10:33:32.658901 21997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:32.645175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:42245 (local address 127.15.82.65:53995)
0929 10:33:32.645625 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:32.645666 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:32.647008 (+ 1342us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:32.647574 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:32.647581 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:32.647939 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:32.648416 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:32.648429 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:32.649265 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:32.649273 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:32.650143 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:32.650152 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:32.650786 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:32.650790 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:32.650961 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:32.651652 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:32.651672 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:32.653671 (+ 1999us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:32.655688 (+ 2017us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:32.655693 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:32.655704 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:32.655956 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:32.656277 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:32.656279 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:32.656280 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:32.656407 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:32.656886 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:32.656891 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:32.657183 (+ 292us) client_negotiation.cc:769] Sending connection context
0929 10:33:32.657343 (+ 160us) client_negotiation.cc:240] Negotiation successful
0929 10:33:32.657592 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":124,"threads_started":1}
I20260929 10:33:32.660243 21995 heartbeater.cc:347] Connected to a master server at 127.15.82.126:42245
I20260929 10:33:32.660491 21995 heartbeater.cc:464] Registering TS with master...
I20260929 10:33:32.660989 21995 heartbeater.cc:511] Master 127.15.82.126:42245 requested a full tablet report, sending...
WARNING: no policy specified for oryx/127.15.82.66@KRBTEST.COM; defaulting to no policy
I20260929 10:33:32.662662 21799 ts_manager.cc:194] Registered new tserver with Master: 068171b1b6514121936f4634de5a8064 (127.15.82.65:45833)
I20260929 10:33:32.663779 21799 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.82.65@KRBTEST.COM'} at 127.15.82.65:53995
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.82.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.82.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/krb5kdc/oryx_127.15.82.66.keytab.
Entry for principal oryx/127.15.82.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/krb5kdc/oryx_127.15.82.66.keytab.
I20260929 10:33:32.682159 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.66:0
--local_ip_for_outbound_sockets=127.15.82.66
--webserver_interface=127.15.82.66
--webserver_port=0
--tserver_master_addrs=127.15.82.126:42245
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/krb5kdc/oryx_127.15.82.66.keytab
--principal=oryx/127.15.82.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:40517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:33:32.794500 22000 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:33:32.794757 22000 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:33:32.794799 22000 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:33:32.798779 22000 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:33:32.798846 22000 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:33:32.798902 22000 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.66
I20260929 10:33:32.804083 22000 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:40517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/krb5kdc/oryx_127.15.82.66.keytab
--principal=oryx/127.15.82.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.82.66
--webserver_port=0
--tserver_master_addrs=127.15.82.126:42245
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22000
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.66
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:33:32.805090 22000 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:33:32.806071 22000 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:33:32.815912 22008 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:32.815893 22006 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:32.815893 22005 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:32.816352 22000 server_base.cc:1034] running on GCE node
I20260929 10:33:32.816823 22000 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:33:32.817512 22000 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:33:32.818729 22000 hybrid_clock.cc:648] HybridClock initialized: now 1790678012818716 us; error 29 us; skew 500 ppm
Sep 29 10:33:32 dist-test-slave-vg89 krb5kdc[21759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678012, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.82.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:33:32.821472 22000 init.cc:377] Logged in from keytab as oryx/127.15.82.66@KRBTEST.COM (short username oryx)
I20260929 10:33:32.822335 22000 webserver.cc:460] Webserver started at http://127.15.82.66:43249/ using document root <none> and password file <none>
I20260929 10:33:32.822921 22000 fs_manager.cc:359] Metadata directory not provided
I20260929 10:33:32.823017 22000 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:33:32.823292 22000 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:33:32.825316 22000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-1/data/instance:
uuid: "9886d2dce2354767a0ee1823c1021808"
format_stamp: "Formatted at 2026-09-29 10:33:32 on dist-test-slave-vg89"
server_key: "4490963b4c67eb9bbe5f694a9151397c"
server_key_iv: "f9efb776d4d2197828ca21e428fe2955"
server_key_version: "encryptionkey@0"
I20260929 10:33:32.825886 22000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-1/wal/instance:
uuid: "9886d2dce2354767a0ee1823c1021808"
format_stamp: "Formatted at 2026-09-29 10:33:32 on dist-test-slave-vg89"
server_key: "4490963b4c67eb9bbe5f694a9151397c"
server_key_iv: "f9efb776d4d2197828ca21e428fe2955"
server_key_version: "encryptionkey@0"
I20260929 10:33:32.829945 22000 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260929 10:33:32.832581 22015 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:32.833829 22000 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 10:33:32.833961 22000 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-1/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-1/wal
uuid: "9886d2dce2354767a0ee1823c1021808"
format_stamp: "Formatted at 2026-09-29 10:33:32 on dist-test-slave-vg89"
server_key: "4490963b4c67eb9bbe5f694a9151397c"
server_key_iv: "f9efb776d4d2197828ca21e428fe2955"
server_key_version: "encryptionkey@0"
I20260929 10:33:32.834115 22000 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:33:32.887079 22000 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:33:32.890391 22000 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:33:32.890636 22000 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:33:32.891247 22000 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:33:32.892041 22000 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:33:32.892107 22000 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:32.892177 22000 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:33:32.892227 22000 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:32.900924 22000 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.66:43097
I20260929 10:33:32.900949 22128 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.66:43097 every 8 connection(s)
I20260929 10:33:32.901777 22000 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
I20260929 10:33:32.909700 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 22000
I20260929 10:33:32.909834 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-1/wal/instance
I20260929 10:33:32.910115 15689 external_mini_cluster.cc:1428] Setting key 6ebabc11664dc1b194754360bb7b1356
Sep 29 10:33:32 dist-test-slave-vg89 krb5kdc[21759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678012, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.82.66@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM
I20260929 10:33:32.915975 21843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:32.903502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:51467 (local address 127.15.82.126:42245)
0929 10:33:32.903643 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:32.903646 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:32.904446 (+ 800us) server_negotiation.cc:407] Connection header received
0929 10:33:32.905828 (+ 1382us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:32.905832 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:32.905889 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:32.905980 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:32.907246 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:32.907857 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:32.908995 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:32.909231 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:32.909344 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:32.912077 (+ 2733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:32.912081 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:32.912084 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:32.912115 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:32.913816 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:32.914320 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:32.914326 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:32.914329 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:32.914453 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:32.914843 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:32.914846 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:32.914847 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:32.915023 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:32.915100 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:32.915753 (+ 653us) server_negotiation.cc:299] Negotiation successful
0929 10:33:32.915852 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260929 10:33:32.917256 22131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:32.903874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:42245 (local address 127.15.82.66:51467)
0929 10:33:32.904298 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:32.904339 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:32.905605 (+ 1266us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:32.906162 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:32.906170 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:32.906612 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:32.907092 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:32.907106 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:32.907981 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:32.907991 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:32.908855 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:32.908865 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:32.909380 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:32.909384 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:32.909573 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:32.910244 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:32.910270 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:32.911912 (+ 1642us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:32.913941 (+ 2029us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:32.913946 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:32.913958 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:32.914215 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:32.914554 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:32.914559 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:32.914563 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:32.914748 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:32.915127 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:32.915136 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:32.915487 (+ 351us) client_negotiation.cc:769] Sending connection context
0929 10:33:32.915710 (+ 223us) client_negotiation.cc:240] Negotiation successful
0929 10:33:32.916095 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":117,"threads_started":1}
I20260929 10:33:32.918605 22129 heartbeater.cc:347] Connected to a master server at 127.15.82.126:42245
I20260929 10:33:32.918861 22129 heartbeater.cc:464] Registering TS with master...
I20260929 10:33:32.919360 22129 heartbeater.cc:511] Master 127.15.82.126:42245 requested a full tablet report, sending...
I20260929 10:33:32.920460 21799 ts_manager.cc:194] Registered new tserver with Master: 9886d2dce2354767a0ee1823c1021808 (127.15.82.66:43097)
WARNING: no policy specified for oryx/127.15.82.67@KRBTEST.COM; defaulting to no policy
I20260929 10:33:32.921034 21799 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.82.66@KRBTEST.COM'} at 127.15.82.66:51467
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.82.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.82.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/krb5kdc/oryx_127.15.82.67.keytab.
Entry for principal oryx/127.15.82.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/krb5kdc/oryx_127.15.82.67.keytab.
I20260929 10:33:32.938961 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.67:0
--local_ip_for_outbound_sockets=127.15.82.67
--webserver_interface=127.15.82.67
--webserver_port=0
--tserver_master_addrs=127.15.82.126:42245
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/krb5kdc/oryx_127.15.82.67.keytab
--principal=oryx/127.15.82.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:40517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:33:33.049752 22134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:33:33.050055 22134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:33:33.050143 22134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:33:33.054214 22134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:33:33.054303 22134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:33:33.054399 22134 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.67
I20260929 10:33:33.059669 22134 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:40517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/krb5kdc/oryx_127.15.82.67.keytab
--principal=oryx/127.15.82.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.82.67
--webserver_port=0
--tserver_master_addrs=127.15.82.126:42245
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22134
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.67
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:33:33.060787 22134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:33:33.061859 22134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:33:33.071714 22142 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:33.071714 22140 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:33.071732 22139 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:33.072115 22134 server_base.cc:1034] running on GCE node
I20260929 10:33:33.072567 22134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:33:33.073278 22134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:33:33.074476 22134 hybrid_clock.cc:648] HybridClock initialized: now 1790678013074444 us; error 54 us; skew 500 ppm
Sep 29 10:33:33 dist-test-slave-vg89 krb5kdc[21759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678013, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.82.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:33:33.077287 22134 init.cc:377] Logged in from keytab as oryx/127.15.82.67@KRBTEST.COM (short username oryx)
I20260929 10:33:33.078143 22134 webserver.cc:460] Webserver started at http://127.15.82.67:34921/ using document root <none> and password file <none>
I20260929 10:33:33.078768 22134 fs_manager.cc:359] Metadata directory not provided
I20260929 10:33:33.078831 22134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:33:33.079001 22134 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:33:33.081037 22134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-2/data/instance:
uuid: "d2e85165e3af490ea649ff60a3d44c65"
format_stamp: "Formatted at 2026-09-29 10:33:33 on dist-test-slave-vg89"
server_key: "e808de97ab18ca7082b0764d9ddd001f"
server_key_iv: "b781639d5ba09f322a4583e3f9d31d6b"
server_key_version: "encryptionkey@0"
I20260929 10:33:33.081561 22134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-2/wal/instance:
uuid: "d2e85165e3af490ea649ff60a3d44c65"
format_stamp: "Formatted at 2026-09-29 10:33:33 on dist-test-slave-vg89"
server_key: "e808de97ab18ca7082b0764d9ddd001f"
server_key_iv: "b781639d5ba09f322a4583e3f9d31d6b"
server_key_version: "encryptionkey@0"
I20260929 10:33:33.085407 22134 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260929 10:33:33.088021 22149 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:33.089354 22134 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 10:33:33.089491 22134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-2/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-2/wal
uuid: "d2e85165e3af490ea649ff60a3d44c65"
format_stamp: "Formatted at 2026-09-29 10:33:33 on dist-test-slave-vg89"
server_key: "e808de97ab18ca7082b0764d9ddd001f"
server_key_iv: "b781639d5ba09f322a4583e3f9d31d6b"
server_key_version: "encryptionkey@0"
I20260929 10:33:33.089604 22134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:33:33.112810 22134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:33:33.116124 22134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:33:33.116350 22134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:33:33.116972 22134 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:33:33.117839 22134 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:33:33.117913 22134 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:33.117983 22134 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:33:33.118031 22134 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:33.127229 22134 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.67:43005
I20260929 10:33:33.127245 22262 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.67:43005 every 8 connection(s)
I20260929 10:33:33.128209 22134 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
I20260929 10:33:33.135517 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 22134
I20260929 10:33:33.135654 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-2/wal/instance
I20260929 10:33:33.135938 15689 external_mini_cluster.cc:1428] Setting key c222f4bd8132e05aa89a5c67b7f72a35
Sep 29 10:33:33 dist-test-slave-vg89 krb5kdc[21759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678013, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.82.67@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM
I20260929 10:33:33.142974 21843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:33.130011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.67:57731 (local address 127.15.82.126:42245)
0929 10:33:33.130143 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:33.130154 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:33.131052 (+ 898us) server_negotiation.cc:407] Connection header received
0929 10:33:33.132463 (+ 1411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:33.132467 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:33.132530 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:33.132633 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:33.133868 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.134564 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.135935 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.136265 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.136372 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:33.139528 (+ 3156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:33.139531 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:33.139533 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:33.139559 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:33.141081 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:33.141530 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:33.141532 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:33.141534 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:33.141609 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:33.141968 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:33.141970 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:33.141971 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:33.142140 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:33.142212 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:33.142743 (+ 531us) server_negotiation.cc:299] Negotiation successful
0929 10:33:33.142852 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260929 10:33:33.143821 22265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:33.130361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:42245 (local address 127.15.82.67:57731)
0929 10:33:33.130885 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:33.130929 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:33.132232 (+ 1303us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:33.132826 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:33.132834 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:33.133211 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:33.133706 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:33.133720 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.134700 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.134703 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:33.135812 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:33.135821 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.136688 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.136693 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:33.136906 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:33.137650 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:33.137681 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:33.139374 (+ 1693us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:33.141180 (+ 1806us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:33.141184 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:33.141197 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:33.141446 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:33.141737 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:33.141740 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:33.141742 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:33.141890 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:33.142226 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:33.142231 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:33.142530 (+ 299us) client_negotiation.cc:769] Sending connection context
0929 10:33:33.142697 (+ 167us) client_negotiation.cc:240] Negotiation successful
0929 10:33:33.142962 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":125,"threads_started":1}
I20260929 10:33:33.145012 22263 heartbeater.cc:347] Connected to a master server at 127.15.82.126:42245
I20260929 10:33:33.145262 22263 heartbeater.cc:464] Registering TS with master...
I20260929 10:33:33.145771 22263 heartbeater.cc:511] Master 127.15.82.126:42245 requested a full tablet report, sending...
I20260929 10:33:33.146842 21799 ts_manager.cc:194] Registered new tserver with Master: d2e85165e3af490ea649ff60a3d44c65 (127.15.82.67:43005)
I20260929 10:33:33.147418 21799 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.82.67@KRBTEST.COM'} at 127.15.82.67:57731
I20260929 10:33:33.150223 15689 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.15.82.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.82.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.82.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/krb5kdc/oryx_127.15.82.125.keytab.
Entry for principal oryx/127.15.82.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/krb5kdc/oryx_127.15.82.125.keytab.
I20260929 10:33:33.177932 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.125:41741
--webserver_interface=127.15.82.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/krb5kdc/oryx_127.15.82.125.keytab
--principal=oryx/127.15.82.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:40517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.82.126:42245,127.15.82.125:41741
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:33:33.290481 22268 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:33:33.290750 22268 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:33:33.290830 22268 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:33:33.294893 22268 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:33:33.294945 22268 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:33:33.294963 22268 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:33:33.295025 22268 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:33:33.295059 22268 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:33:33.300450 22268 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:40517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.15.82.126:42245,127.15.82.125:41741
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/krb5kdc/oryx_127.15.82.125.keytab
--principal=oryx/127.15.82.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.125:41741
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.15.82.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22268
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
W20260929 10:33:33.300657 22268 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.15.82.126:42245,127.15.82.125:41741'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260929 10:33:33.301517 22268 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:33:33.302632 22268 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:33:33.311010 22273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:33.311044 22274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:33.311045 22276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:33.311363 22268 server_base.cc:1034] running on GCE node
I20260929 10:33:33.312001 22268 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:33:33.313061 22268 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:33:33.314245 22268 hybrid_clock.cc:648] HybridClock initialized: now 1790678013314234 us; error 31 us; skew 500 ppm
Sep 29 10:33:33 dist-test-slave-vg89 krb5kdc[21759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678013, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.82.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:33:33.316951 22268 init.cc:377] Logged in from keytab as oryx/127.15.82.125@KRBTEST.COM (short username oryx)
I20260929 10:33:33.317854 22268 webserver.cc:460] Webserver started at http://127.15.82.125:33307/ using document root <none> and password file <none>
I20260929 10:33:33.318464 22268 fs_manager.cc:359] Metadata directory not provided
I20260929 10:33:33.318552 22268 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:33:33.318778 22268 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:33:33.320792 22268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/master-1/data/instance:
uuid: "ce1288d49e714ca69abeca37c76aebae"
format_stamp: "Formatted at 2026-09-29 10:33:33 on dist-test-slave-vg89"
server_key: "b56b85e00244320e65c67ef27e4ce96d"
server_key_iv: "0930748c2adffea1b151752603bd523b"
server_key_version: "encryptionkey@0"
I20260929 10:33:33.321439 22268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/master-1/wal/instance:
uuid: "ce1288d49e714ca69abeca37c76aebae"
format_stamp: "Formatted at 2026-09-29 10:33:33 on dist-test-slave-vg89"
server_key: "b56b85e00244320e65c67ef27e4ce96d"
server_key_iv: "0930748c2adffea1b151752603bd523b"
server_key_version: "encryptionkey@0"
I20260929 10:33:33.325075 22268 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260929 10:33:33.327390 22283 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:33.328569 22268 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 10:33:33.328713 22268 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/master-1/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/master-1/wal
uuid: "ce1288d49e714ca69abeca37c76aebae"
format_stamp: "Formatted at 2026-09-29 10:33:33 on dist-test-slave-vg89"
server_key: "b56b85e00244320e65c67ef27e4ce96d"
server_key_iv: "0930748c2adffea1b151752603bd523b"
server_key_version: "encryptionkey@0"
I20260929 10:33:33.328820 22268 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:33:33.343813 22268 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:33:33.346510 22268 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:33:33.346706 22268 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:33:33.353505 22268 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.125:41741
I20260929 10:33:33.353777 22335 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.125:41741 every 8 connection(s)
I20260929 10:33:33.354573 22268 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/master-1/data/info.pb
I20260929 10:33:33.357460 22336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:33:33.361621 22336 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.82.126" port: 42245 } has no permanent_uuid. Determining permanent_uuid...
I20260929 10:33:33.364331 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 22268
Sep 29 10:33:33 dist-test-slave-vg89 krb5kdc[21759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678013, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.82.125@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM
Sep 29 10:33:33 dist-test-slave-vg89 krb5kdc[21759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678012, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.82.125@KRBTEST.COM
I20260929 10:33:33.376156 21843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:33.363697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37036 (local address 127.15.82.126:42245)
0929 10:33:33.363846 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:33.363851 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:33.364742 (+ 891us) server_negotiation.cc:407] Connection header received
0929 10:33:33.365849 (+ 1107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:33.365853 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:33.365910 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:33.366011 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:33.367318 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.368078 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.369177 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.369395 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.369514 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:33.372228 (+ 2714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:33.372233 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:33.372236 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:33.372268 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:33.373944 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:33.374480 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:33.374483 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:33.374485 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:33.374569 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:33.374935 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:33.374939 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:33.374940 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:33.375182 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:33.375260 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:33.375869 (+ 609us) server_negotiation.cc:299] Negotiation successful
0929 10:33:33.375996 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260929 10:33:33.377013 22338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:33.364100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:42245 (local address 127.0.0.1:37036)
0929 10:33:33.364596 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:33.364647 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:33.365651 (+ 1004us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:33.366091 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:33.366102 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:33.366552 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:33.367147 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:33.367177 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.368217 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.368221 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:33.369042 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:33.369050 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.369664 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.369668 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:33.369885 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:33.370533 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:33.370548 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:33.372055 (+ 1507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:33.374084 (+ 2029us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:33.374091 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:33.374099 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:33.374340 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:33.374682 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:33.374685 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:33.374688 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:33.374840 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:33.375322 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:33.375328 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:33.375608 (+ 280us) client_negotiation.cc:769] Sending connection context
0929 10:33:33.375796 (+ 188us) client_negotiation.cc:240] Negotiation successful
0929 10:33:33.376085 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"mutex_wait_us":30,"spinlock_wait_cycles":6272,"thread_start_us":143,"threads_started":1}
I20260929 10:33:33.377199 22340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:33.365404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42264 (local address 127.15.82.125:41741)
0929 10:33:33.365869 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:33.365879 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:33.365916 (+ 37us) server_negotiation.cc:407] Connection header received
0929 10:33:33.366054 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:33.366059 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:33.366257 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:33.366446 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:33.367411 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.368318 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.369464 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.370014 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.370134 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:33.373027 (+ 2893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:33.373033 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:33.373063 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:33.373095 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:33.375070 (+ 1975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:33.375511 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:33.375523 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:33.375524 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:33.375601 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:33.375918 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:33.375921 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:33.375923 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:33.376211 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:33.376411 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:33.376781 (+ 370us) server_negotiation.cc:299] Negotiation successful
0929 10:33:33.376916 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":48,"server-negotiator.queue_time_us":377,"spinlock_wait_cycles":4352,"thread_start_us":120,"threads_started":1}
I20260929 10:33:33.378341 22336 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.82.125" port: 41741 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 10:33:33 dist-test-slave-vg89 krb5kdc[21759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678013, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.82.125@KRBTEST.COM for oryx/127.15.82.125@KRBTEST.COM
I20260929 10:33:33.386332 22338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:33.378842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.125:41741 (local address 127.0.0.1:42274)
0929 10:33:33.378923 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:33.378937 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:33.379007 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:33.379244 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:33.379246 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:33.379419 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:33.379514 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:33.379519 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.380196 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.380198 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:33.380839 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:33.380846 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.381693 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.381695 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:33.381764 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:33.382118 (+ 354us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:33.382128 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:33.383364 (+ 1236us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:33.384971 (+ 1607us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:33.384977 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:33.384980 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:33.385154 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:33.385412 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:33.385414 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:33.385416 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:33.385486 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:33.385836 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:33.385839 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:33.385920 (+ 81us) client_negotiation.cc:769] Sending connection context
0929 10:33:33.386016 (+ 96us) client_negotiation.cc:240] Negotiation successful
0929 10:33:33.386141 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260929 10:33:33.386336 22340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:33.378896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42274 (local address 127.15.82.125:41741)
0929 10:33:33.379000 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:33.379006 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:33.379019 (+ 13us) server_negotiation.cc:407] Connection header received
0929 10:33:33.379089 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:33.379092 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:33.379131 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:33.379220 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:33.379612 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.380096 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.380944 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.381598 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.381683 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:33.383478 (+ 1795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:33.383481 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:33.383483 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:33.383502 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:33.384864 (+ 1362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:33.385240 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:33.385246 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:33.385248 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:33.385319 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:33.385573 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:33.385575 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:33.385577 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:33.385744 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:33.385846 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:33.386068 (+ 222us) server_negotiation.cc:299] Negotiation successful
0929 10:33:33.386203 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":36,"server-negotiator.queue_time_us":26}
I20260929 10:33:33.389034 22336 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce1288d49e714ca69abeca37c76aebae: Bootstrap starting.
I20260929 10:33:33.391264 22336 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce1288d49e714ca69abeca37c76aebae: Neither blocks nor log segments found. Creating new log.
I20260929 10:33:33.391916 22336 log.cc:824] T 00000000000000000000000000000000 P ce1288d49e714ca69abeca37c76aebae: Log is configured to *not* fsync() on all Append() calls
I20260929 10:33:33.393746 22336 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce1288d49e714ca69abeca37c76aebae: No bootstrap required, opened a new log
I20260929 10:33:33.396680 22336 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce1288d49e714ca69abeca37c76aebae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b919a4b6dcc4702b515509aa1dd2164" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 42245 } } peers { permanent_uuid: "ce1288d49e714ca69abeca37c76aebae" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 41741 } }
I20260929 10:33:33.396867 22336 raft_consensus.cc:374] T 00000000000000000000000000000000 P ce1288d49e714ca69abeca37c76aebae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:33:33.396898 22336 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce1288d49e714ca69abeca37c76aebae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce1288d49e714ca69abeca37c76aebae, State: Initialized, Role: FOLLOWER
I20260929 10:33:33.397238 22336 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce1288d49e714ca69abeca37c76aebae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b919a4b6dcc4702b515509aa1dd2164" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 42245 } } peers { permanent_uuid: "ce1288d49e714ca69abeca37c76aebae" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 41741 } }
I20260929 10:33:33.397616 22342 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce1288d49e714ca69abeca37c76aebae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b919a4b6dcc4702b515509aa1dd2164" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 42245 } } peers { permanent_uuid: "ce1288d49e714ca69abeca37c76aebae" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 41741 } } }
I20260929 10:33:33.397750 22342 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce1288d49e714ca69abeca37c76aebae [sys.catalog]: This master's current role is: FOLLOWER
I20260929 10:33:33.398319 22336 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce1288d49e714ca69abeca37c76aebae [sys.catalog]: configured and running, proceeding with master startup.
W20260929 10:33:33.402733 22353 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ce1288d49e714ca69abeca37c76aebae: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 10:33:33.402817 22353 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 10:33:33.404531 22268 master_runner.cc:428] Detected that this master ce1288d49e714ca69abeca37c76aebae is joining an existing cluster
I20260929 10:33:33.404601 22268 master_runner.cc:432] Initiating AddMaster RPC to add 127.15.82.125:41741
I20260929 10:33:33.414891 22340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:33.406905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42276 (local address 127.15.82.125:41741)
0929 10:33:33.407090 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:33.407095 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:33.407298 (+ 203us) server_negotiation.cc:407] Connection header received
0929 10:33:33.407345 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:33.407348 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:33.407390 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:33.407481 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:33.408008 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.408524 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.409627 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.409923 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.409974 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:33.411363 (+ 1389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:33.411368 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:33.411374 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:33.411396 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:33.413274 (+ 1878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:33.413675 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:33.413678 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:33.413680 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:33.413757 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:33.414022 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:33.414025 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:33.414027 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:33.414241 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:33.414337 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:33.414607 (+ 270us) server_negotiation.cc:299] Negotiation successful
0929 10:33:33.414712 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
I20260929 10:33:33.415381 22360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:33.406863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.125:41741 (local address 127.0.0.1:42276)
0929 10:33:33.407140 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:33.407155 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:33.407273 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:33.407506 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:33.407509 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:33.407733 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:33.407897 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:33.407907 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.408633 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.408636 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:33.409510 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:33.409519 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.410085 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.410087 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:33.410138 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:33.410723 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:33.410738 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:33.411205 (+ 467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:33.413375 (+ 2170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:33.413381 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:33.413385 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:33.413574 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:33.413849 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:33.413852 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:33.413854 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:33.413930 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:33.414326 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:33.414329 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:33.414427 (+ 98us) client_negotiation.cc:769] Sending connection context
0929 10:33:33.415121 (+ 694us) client_negotiation.cc:240] Negotiation successful
0929 10:33:33.415251 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":73,"threads_started":1}
I20260929 10:33:33.415414 21843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:33.406539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37048 (local address 127.15.82.126:42245)
0929 10:33:33.406693 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:33.406697 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:33.406854 (+ 157us) server_negotiation.cc:407] Connection header received
0929 10:33:33.407047 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:33.407050 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:33.407103 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:33.407160 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:33.407920 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.408760 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.409849 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.410187 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.410300 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:33.411902 (+ 1602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:33.411906 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:33.411909 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:33.411936 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:33.413604 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:33.414077 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:33.414080 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:33.414082 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:33.414157 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:33.414467 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:33.414470 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:33.414472 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:33.414698 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:33.414788 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:33.415131 (+ 343us) server_negotiation.cc:299] Negotiation successful
0929 10:33:33.415249 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 10:33:33.415995 22359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:33.406453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:42245 (local address 127.0.0.1:37048)
0929 10:33:33.406775 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:33.406788 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:33.406924 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:33.407320 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:33.407323 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:33.407607 (+ 284us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:33.407760 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:33.407767 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.408873 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.408876 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:33.409729 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:33.409737 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.410703 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.410710 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:33.410820 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:33.411308 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:33.411326 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:33.411747 (+ 421us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:33.413749 (+ 2002us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:33.413752 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:33.413754 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:33.413991 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:33.414260 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:33.414263 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:33.414265 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:33.414363 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:33.414794 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:33.414797 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:33.414897 (+ 100us) client_negotiation.cc:769] Sending connection context
0929 10:33:33.415701 (+ 804us) client_negotiation.cc:240] Negotiation successful
0929 10:33:33.415836 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"mutex_wait_us":1,"thread_start_us":100,"threads_started":1}
I20260929 10:33:33.425791 22342 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce1288d49e714ca69abeca37c76aebae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
Sep 29 10:33:33 dist-test-slave-vg89 krb5kdc[21759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678012, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.82.126@KRBTEST.COM for oryx/127.15.82.125@KRBTEST.COM
I20260929 10:33:33.425962 22342 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce1288d49e714ca69abeca37c76aebae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b919a4b6dcc4702b515509aa1dd2164" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 42245 } } peers { permanent_uuid: "ce1288d49e714ca69abeca37c76aebae" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 41741 } }
I20260929 10:33:33.426949 22342 leader_election.cc:288] T 00000000000000000000000000000000 P ce1288d49e714ca69abeca37c76aebae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b919a4b6dcc4702b515509aa1dd2164 (127.15.82.126:42245)
I20260929 10:33:33.427412 21814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ce1288d49e714ca69abeca37c76aebae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b919a4b6dcc4702b515509aa1dd2164" is_pre_election: true
I20260929 10:33:33.428092 22284 leader_election.cc:302] T 00000000000000000000000000000000 P ce1288d49e714ca69abeca37c76aebae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: ce1288d49e714ca69abeca37c76aebae; no voters: 2b919a4b6dcc4702b515509aa1dd2164
I20260929 10:33:33.428519 22342 raft_consensus.cc:2726] T 00000000000000000000000000000000 P ce1288d49e714ca69abeca37c76aebae [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260929 10:33:33.430181 22340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:33.420305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42278 (local address 127.15.82.125:41741)
0929 10:33:33.420426 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:33.420430 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:33.420803 (+ 373us) server_negotiation.cc:407] Connection header received
0929 10:33:33.421010 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:33.421012 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:33.421054 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:33.421119 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:33.421896 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.422365 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.423646 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.423846 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.423923 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:33.426587 (+ 2664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:33.426597 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:33.426600 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:33.426620 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:33.428251 (+ 1631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:33.428718 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:33.428721 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:33.428723 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:33.428805 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:33.429056 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:33.429059 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:33.429061 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:33.429275 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:33.429370 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:33.429884 (+ 514us) server_negotiation.cc:299] Negotiation successful
0929 10:33:33.429990 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260929 10:33:33.430550 22361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:33.420257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.125:41741 (local address 127.0.0.1:42278)
0929 10:33:33.420656 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:33.420713 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:33.420881 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:33.421174 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:33.421180 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:33.421649 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:33.421788 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:33.421795 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.422527 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.422530 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:33.423539 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:33.423550 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.423950 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.423953 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:33.424082 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:33.424679 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:33.424693 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:33.426363 (+ 1670us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:33.428341 (+ 1978us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:33.428351 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:33.428357 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:33.428620 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:33.428894 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:33.428896 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:33.428897 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:33.428972 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:33.429377 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:33.429382 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:33.429487 (+ 105us) client_negotiation.cc:769] Sending connection context
0929 10:33:33.429827 (+ 340us) client_negotiation.cc:240] Negotiation successful
0929 10:33:33.430341 (+ 514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":102,"threads_started":1}
I20260929 10:33:33.431895 21799 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.15.82.125:41741: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "ce1288d49e714ca69abeca37c76aebae"
member_type: NON_VOTER
last_known_addr {
host: "127.15.82.125"
port: 41741
}
attrs {
promote: true
}
}
dest_uuid: "2b919a4b6dcc4702b515509aa1dd2164"
cas_config_opid_index: -1
I20260929 10:33:33.432629 21799 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2b919a4b6dcc4702b515509aa1dd2164 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2b919a4b6dcc4702b515509aa1dd2164" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 42245 } } peers { permanent_uuid: "ce1288d49e714ca69abeca37c76aebae" member_type: NON_VOTER last_known_addr { host: "127.15.82.125" port: 41741 } attrs { promote: true } }
I20260929 10:33:33.434855 22364 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2b919a4b6dcc4702b515509aa1dd2164 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER ce1288d49e714ca69abeca37c76aebae (127.15.82.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2b919a4b6dcc4702b515509aa1dd2164" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 42245 } } peers { permanent_uuid: "ce1288d49e714ca69abeca37c76aebae" member_type: NON_VOTER last_known_addr { host: "127.15.82.125" port: 41741 } attrs { promote: true } } }
I20260929 10:33:33.435221 22310 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce1288d49e714ca69abeca37c76aebae [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:33:33.436662 22364 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.15.82.125:41741
I20260929 10:33:33.436884 22310 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ce1288d49e714ca69abeca37c76aebae [term 1 FOLLOWER]: Refusing update from remote peer 2b919a4b6dcc4702b515509aa1dd2164: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260929 10:33:33.437218 22268 master.cc:537] Master@127.15.82.125:41741 shutting down...
I20260929 10:33:33.438551 22365 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2b919a4b6dcc4702b515509aa1dd2164 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce1288d49e714ca69abeca37c76aebae" member_type: NON_VOTER last_known_addr { host: "127.15.82.125" port: 41741 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260929 10:33:33.440634 21788 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
WARNING: no policy specified for oryx/127.15.82.124@KRBTEST.COM; defaulting to no policy
W20260929 10:33:33.441570 21788 consensus_peers.cc:487] T 00000000000000000000000000000000 P 2b919a4b6dcc4702b515509aa1dd2164 -> Peer ce1288d49e714ca69abeca37c76aebae (127.15.82.125:41741): Couldn't send request to peer ce1288d49e714ca69abeca37c76aebae. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260929 10:33:33.442039 22366 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b919a4b6dcc4702b515509aa1dd2164 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "2b919a4b6dcc4702b515509aa1dd2164" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2b919a4b6dcc4702b515509aa1dd2164" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 42245 } } peers { permanent_uuid: "ce1288d49e714ca69abeca37c76aebae" member_type: NON_VOTER last_known_addr { host: "127.15.82.125" port: 41741 } attrs { promote: true } } }
I20260929 10:33:33.442153 22366 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b919a4b6dcc4702b515509aa1dd2164 [sys.catalog]: This master's current role is: LEADER
I20260929 10:33:33.442615 22268 raft_consensus.cc:2229] T 00000000000000000000000000000000 P ce1288d49e714ca69abeca37c76aebae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:33:33.443028 22268 raft_consensus.cc:2258] T 00000000000000000000000000000000 P ce1288d49e714ca69abeca37c76aebae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:33:33.443137 22268 tablet_replica.cc:330] T 00000000000000000000000000000000 P ce1288d49e714ca69abeca37c76aebae: stopping tablet replica
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.82.124@KRBTEST.COM" created.
I20260929 10:33:33.448335 22268 master.cc:552] Master@127.15.82.125:41741 shutdown complete.
I20260929 10:33:33.448392 22268 master_runner.cc:305] Clearing existing system tablet
I20260929 10:33:33.448654 22268 fs_manager.cc:359] Metadata directory not provided
I20260929 10:33:33.448734 22268 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:33:33.450306 22268 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 10:33:33.451330 22371 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:33.451555 22268 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:33:33.451666 22268 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:33:33.451697 22268 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/master-1/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/master-1/wal
uuid: "ce1288d49e714ca69abeca37c76aebae"
format_stamp: "Formatted at 2026-09-29 10:33:33 on dist-test-slave-vg89"
server_key: "b56b85e00244320e65c67ef27e4ce96d"
server_key_iv: "0930748c2adffea1b151752603bd523b"
server_key_version: "encryptionkey@0"
I20260929 10:33:33.453804 22268 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P ce1288d49e714ca69abeca37c76aebae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:33:33.456817 22268 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P ce1288d49e714ca69abeca37c76aebae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260929 10:33:33.456907 22268 log.cc:1196] T 00000000000000000000000000000000 P ce1288d49e714ca69abeca37c76aebae: Deleting WAL directory at /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260929 10:33:33.457140 22268 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P ce1288d49e714ca69abeca37c76aebae: Deleting consensus metadata
I20260929 10:33:33.457320 22268 master_runner.cc:315] Copying system tablet from 127.15.82.126:42245
I20260929 10:33:33.458493 22268 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P ce1288d49e714ca69abeca37c76aebae: tablet copy: Beginning tablet copy session from remote peer at address 127.15.82.126:42245
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.82.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/krb5kdc/oryx_127.15.82.124.keytab.
Entry for principal oryx/127.15.82.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/krb5kdc/oryx_127.15.82.124.keytab.
I20260929 10:33:33.462545 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.124:41423
--webserver_interface=127.15.82.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/krb5kdc/oryx_127.15.82.124.keytab
--principal=oryx/127.15.82.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:40517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.82.126:42245,127.15.82.125:41741,127.15.82.124:41423
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 10:33:33.467818 21843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:33.459169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37058 (local address 127.15.82.126:42245)
0929 10:33:33.459306 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:33.459311 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:33.459475 (+ 164us) server_negotiation.cc:407] Connection header received
0929 10:33:33.459581 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:33.459584 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:33.459633 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:33.459753 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:33.460239 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.461059 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.462182 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.462462 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.462567 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:33.464129 (+ 1562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:33.464134 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:33.464138 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:33.464184 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:33.465816 (+ 1632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:33.466242 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:33.466248 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:33.466253 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:33.466345 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:33.466687 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:33.466693 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:33.466698 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:33.466949 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:33.467028 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:33.467429 (+ 401us) server_negotiation.cc:299] Negotiation successful
0929 10:33:33.467569 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260929 10:33:33.468532 22377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:33.459054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:42245 (local address 127.0.0.1:37058)
0929 10:33:33.459354 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:33.459369 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:33.459497 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:33.459805 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:33.459811 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:33.460015 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:33.460109 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:33.460115 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.461174 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.461176 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:33.462053 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:33.462063 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.462591 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.462594 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:33.462679 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:33.463600 (+ 921us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:33.463612 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:33.463922 (+ 310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:33.465931 (+ 2009us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:33.465936 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:33.465938 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:33.466141 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:33.466474 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:33.466479 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:33.466484 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:33.466580 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:33.467104 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:33.467109 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:33.467233 (+ 124us) client_negotiation.cc:769] Sending connection context
0929 10:33:33.468063 (+ 830us) client_negotiation.cc:240] Negotiation successful
0929 10:33:33.468253 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":151,"threads_started":1}
I20260929 10:33:33.469133 21824 tablet_copy_service.cc:140] P 2b919a4b6dcc4702b515509aa1dd2164: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer ce1288d49e714ca69abeca37c76aebae ({username='oryx', principal='oryx/127.15.82.125@KRBTEST.COM'} at 127.0.0.1:37058)
I20260929 10:33:33.469290 21824 tablet_copy_service.cc:161] P 2b919a4b6dcc4702b515509aa1dd2164: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer ce1288d49e714ca69abeca37c76aebae at {username='oryx', principal='oryx/127.15.82.125@KRBTEST.COM'} at 127.0.0.1:37058: session id = ce1288d49e714ca69abeca37c76aebae-00000000000000000000000000000000
I20260929 10:33:33.471199 21824 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 2b919a4b6dcc4702b515509aa1dd2164: Tablet Copy: opened 0 blocks and 1 log segments
I20260929 10:33:33.473285 22268 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260929 10:33:33.476090 22268 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P ce1288d49e714ca69abeca37c76aebae: tablet copy: Starting download of 0 data blocks...
I20260929 10:33:33.476293 22268 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P ce1288d49e714ca69abeca37c76aebae: tablet copy: Starting download of 1 WAL segments...
I20260929 10:33:33.478061 22268 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P ce1288d49e714ca69abeca37c76aebae: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260929 10:33:33.479925 21824 tablet_copy_service.cc:342] P 2b919a4b6dcc4702b515509aa1dd2164: Request end of tablet copy session ce1288d49e714ca69abeca37c76aebae-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.15.82.125@KRBTEST.COM'} at 127.0.0.1:37058
I20260929 10:33:33.480026 21824 tablet_copy_service.cc:433] P 2b919a4b6dcc4702b515509aa1dd2164: ending tablet copy session ce1288d49e714ca69abeca37c76aebae-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer ce1288d49e714ca69abeca37c76aebae
I20260929 10:33:33.487720 22268 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:33:33.489588 22379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:33.491405 22380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:33.491566 22382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:33.491804 22268 server_base.cc:1034] running on GCE node
I20260929 10:33:33.491955 22268 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:33:33.492254 22268 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:33:33.493391 22268 hybrid_clock.cc:648] HybridClock initialized: now 1790678013493376 us; error 27 us; skew 500 ppm
Sep 29 10:33:33 dist-test-slave-vg89 krb5kdc[21759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678013, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.82.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:33:33.495039 22268 init.cc:377] Logged in from keytab as oryx/127.15.82.125@KRBTEST.COM (short username oryx)
I20260929 10:33:33.495512 22268 webserver.cc:460] Webserver started at http://127.15.82.125:40023/ using document root <none> and password file <none>
I20260929 10:33:33.495689 22268 fs_manager.cc:359] Metadata directory not provided
I20260929 10:33:33.495724 22268 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:33:33.497004 22268 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:33:33.498062 22389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:33.498687 22268 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:33:33.498796 22268 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/master-1/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/master-1/wal
uuid: "ce1288d49e714ca69abeca37c76aebae"
format_stamp: "Formatted at 2026-09-29 10:33:33 on dist-test-slave-vg89"
server_key: "b56b85e00244320e65c67ef27e4ce96d"
server_key_iv: "0930748c2adffea1b151752603bd523b"
server_key_version: "encryptionkey@0"
I20260929 10:33:33.498932 22268 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:33:33.527966 22268 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:33:33.530321 22268 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:33:33.535262 22268 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.125:41741
I20260929 10:33:33.535305 22441 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.125:41741 every 8 connection(s)
I20260929 10:33:33.535976 22268 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/master-1/data/info.pb
I20260929 10:33:33.538062 22442 sys_catalog.cc:263] Verifying existing consensus state
I20260929 10:33:33.539145 22442 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce1288d49e714ca69abeca37c76aebae: Bootstrap starting.
I20260929 10:33:33.549937 22442 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce1288d49e714ca69abeca37c76aebae: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 10:33:33.550293 22442 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce1288d49e714ca69abeca37c76aebae: Bootstrap complete.
I20260929 10:33:33.550961 22442 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce1288d49e714ca69abeca37c76aebae [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2b919a4b6dcc4702b515509aa1dd2164" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 42245 } } peers { permanent_uuid: "ce1288d49e714ca69abeca37c76aebae" member_type: NON_VOTER last_known_addr { host: "127.15.82.125" port: 41741 } attrs { promote: true } }
I20260929 10:33:33.551110 22442 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce1288d49e714ca69abeca37c76aebae [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ce1288d49e714ca69abeca37c76aebae, State: Initialized, Role: LEARNER
I20260929 10:33:33.551236 22442 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce1288d49e714ca69abeca37c76aebae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2b919a4b6dcc4702b515509aa1dd2164" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 42245 } } peers { permanent_uuid: "ce1288d49e714ca69abeca37c76aebae" member_type: NON_VOTER last_known_addr { host: "127.15.82.125" port: 41741 } attrs { promote: true } }
I20260929 10:33:33.551597 22445 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce1288d49e714ca69abeca37c76aebae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2b919a4b6dcc4702b515509aa1dd2164" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 42245 } } peers { permanent_uuid: "ce1288d49e714ca69abeca37c76aebae" member_type: NON_VOTER last_known_addr { host: "127.15.82.125" port: 41741 } attrs { promote: true } } }
I20260929 10:33:33.551719 22442 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce1288d49e714ca69abeca37c76aebae [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:33:33.551788 22445 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce1288d49e714ca69abeca37c76aebae [sys.catalog]: This master's current role is: LEARNER
I20260929 10:33:33.554841 22456 catalog_manager.cc:1259] Loaded cluster ID: 1f355f33ab054e5c8cc0fe255d71c834
I20260929 10:33:33.554910 22456 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ce1288d49e714ca69abeca37c76aebae: loading cluster ID for follower catalog manager: success
I20260929 10:33:33.556970 22456 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ce1288d49e714ca69abeca37c76aebae: acquiring CA information for follower catalog manager: success
I20260929 10:33:33.558306 22456 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ce1288d49e714ca69abeca37c76aebae: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260929 10:33:33.601127 22378 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:33:33.601418 22378 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:33:33.601501 22378 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:33:33.605528 22378 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:33:33.605619 22378 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:33:33.605657 22378 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:33:33.605712 22378 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:33:33.605749 22378 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:33:33.611425 22378 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:40517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.15.82.126:42245,127.15.82.125:41741,127.15.82.124:41423
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/krb5kdc/oryx_127.15.82.124.keytab
--principal=oryx/127.15.82.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.124:41423
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.15.82.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22378
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:33:33.612514 22378 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:33:33.613648 22378 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:33:33.622282 22462 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:33.622288 22464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:33.622340 22378 server_base.cc:1034] running on GCE node
W20260929 10:33:33.622340 22461 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:33.623121 22378 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:33:33.624253 22378 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:33:33.625439 22378 hybrid_clock.cc:648] HybridClock initialized: now 1790678013625423 us; error 30 us; skew 500 ppm
Sep 29 10:33:33 dist-test-slave-vg89 krb5kdc[21759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678013, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.82.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:33:33.628261 22378 init.cc:377] Logged in from keytab as oryx/127.15.82.124@KRBTEST.COM (short username oryx)
I20260929 10:33:33.629123 22378 webserver.cc:460] Webserver started at http://127.15.82.124:41487/ using document root <none> and password file <none>
I20260929 10:33:33.629693 22378 fs_manager.cc:359] Metadata directory not provided
I20260929 10:33:33.629781 22378 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:33:33.630012 22378 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:33:33.632089 22378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/master-2/data/instance:
uuid: "9df9b9ceadd74c56bc5c0242855d0777"
format_stamp: "Formatted at 2026-09-29 10:33:33 on dist-test-slave-vg89"
server_key: "f8399c33eac1e58e8bd44adf477a914b"
server_key_iv: "bcd1b025f9186a5fe8386736ed50351d"
server_key_version: "encryptionkey@0"
I20260929 10:33:33.632725 22378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/master-2/wal/instance:
uuid: "9df9b9ceadd74c56bc5c0242855d0777"
format_stamp: "Formatted at 2026-09-29 10:33:33 on dist-test-slave-vg89"
server_key: "f8399c33eac1e58e8bd44adf477a914b"
server_key_iv: "bcd1b025f9186a5fe8386736ed50351d"
server_key_version: "encryptionkey@0"
I20260929 10:33:33.636432 22378 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260929 10:33:33.638936 22471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:33.640182 22378 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 10:33:33.640316 22378 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/master-2/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/master-2/wal
uuid: "9df9b9ceadd74c56bc5c0242855d0777"
format_stamp: "Formatted at 2026-09-29 10:33:33 on dist-test-slave-vg89"
server_key: "f8399c33eac1e58e8bd44adf477a914b"
server_key_iv: "bcd1b025f9186a5fe8386736ed50351d"
server_key_version: "encryptionkey@0"
I20260929 10:33:33.640429 22378 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:33:33.652745 22378 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:33:33.655727 22378 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:33:33.655880 22378 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:33:33.662458 22378 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.124:41423
I20260929 10:33:33.662776 22523 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.124:41423 every 8 connection(s)
I20260929 10:33:33.663453 22378 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/master-2/data/info.pb
I20260929 10:33:33.666260 22524 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:33:33.667608 21995 heartbeater.cc:503] Master 127.15.82.126:42245 was elected leader, sending a full tablet report...
I20260929 10:33:33.670315 22524 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.82.126" port: 42245 } has no permanent_uuid. Determining permanent_uuid...
I20260929 10:33:33.671255 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 22378
Sep 29 10:33:33 dist-test-slave-vg89 krb5kdc[21759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678013, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.82.124@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM
Sep 29 10:33:33 dist-test-slave-vg89 krb5kdc[21759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678012, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.82.124@KRBTEST.COM
I20260929 10:33:33.684932 21843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:33.672562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37062 (local address 127.15.82.126:42245)
0929 10:33:33.672711 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:33.672716 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:33.673266 (+ 550us) server_negotiation.cc:407] Connection header received
0929 10:33:33.674029 (+ 763us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:33.674033 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:33.674083 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:33.674191 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:33.675367 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.676198 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.677606 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.677948 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.678054 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:33.680840 (+ 2786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:33.680846 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:33.680852 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:33.680888 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:33.682540 (+ 1652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:33.683078 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:33.683080 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:33.683081 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:33.683153 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:33.683571 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:33.683575 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:33.683577 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:33.683845 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:33.683919 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:33.684584 (+ 665us) server_negotiation.cc:299] Negotiation successful
0929 10:33:33.684725 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 10:33:33.685892 22527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:33.672666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:42245 (local address 127.0.0.1:37062)
0929 10:33:33.673120 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:33.673168 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:33.673829 (+ 661us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:33.674299 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:33.674306 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:33.674835 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:33.675204 (+ 369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:33.675229 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.676323 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.676326 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:33.677478 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:33.677487 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.678103 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.678106 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:33.678264 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:33.679040 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:33.679059 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:33.680676 (+ 1617us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:33.682638 (+ 1962us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:33.682648 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:33.682661 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:33.682983 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:33.683265 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:33.683269 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:33.683272 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:33.683468 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:33.683943 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:33.683952 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:33.684269 (+ 317us) client_negotiation.cc:769] Sending connection context
0929 10:33:33.684528 (+ 259us) client_negotiation.cc:240] Negotiation successful
0929 10:33:33.684850 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"mutex_wait_us":57,"thread_start_us":146,"threads_started":1}
I20260929 10:33:33.685892 22526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:33.672667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38072 (local address 127.15.82.124:41423)
0929 10:33:33.673073 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:33.673088 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:33.673119 (+ 31us) server_negotiation.cc:407] Connection header received
0929 10:33:33.673813 (+ 694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:33.673831 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:33.674072 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:33.674245 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:33.675251 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.676475 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.677601 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.678166 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.678284 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:33.681235 (+ 2951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:33.681243 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:33.681258 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:33.681284 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:33.683356 (+ 2072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:33.683804 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:33.683808 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:33.683809 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:33.683874 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:33.684146 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:33.684149 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:33.684151 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:33.684417 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:33.684610 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:33.684812 (+ 202us) server_negotiation.cc:299] Negotiation successful
0929 10:33:33.684926 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":263,"thread_start_us":130,"threads_started":1}
I20260929 10:33:33.687216 22524 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.82.125" port: 41741 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 10:33:33 dist-test-slave-vg89 krb5kdc[21759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678013, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.82.124@KRBTEST.COM for oryx/127.15.82.125@KRBTEST.COM
I20260929 10:33:33.695369 22527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:33.687718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.125:41741 (local address 127.0.0.1:42282)
0929 10:33:33.687949 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:33.687965 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:33.688039 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:33.688325 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:33.688327 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:33.688539 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:33.688640 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:33.688645 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.689629 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.689632 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:33.690185 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:33.690192 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.690728 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.690732 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:33.690827 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:33.691204 (+ 377us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:33.691214 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:33.692385 (+ 1171us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:33.693923 (+ 1538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:33.693927 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:33.693929 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:33.694113 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:33.694389 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:33.694393 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:33.694420 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:33.694509 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:33.694876 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:33.694879 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:33.694976 (+ 97us) client_negotiation.cc:769] Sending connection context
0929 10:33:33.695078 (+ 102us) client_negotiation.cc:240] Negotiation successful
0929 10:33:33.695210 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141}
I20260929 10:33:33.695407 22528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:33.687851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42282 (local address 127.15.82.125:41741)
0929 10:33:33.688118 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:33.688122 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:33.688134 (+ 12us) server_negotiation.cc:407] Connection header received
0929 10:33:33.688181 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:33.688183 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:33.688220 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:33.688312 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:33.688749 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.689514 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.690301 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.690614 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.690731 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:33.692518 (+ 1787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:33.692521 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:33.692524 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:33.692554 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:33.693817 (+ 1263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:33.694216 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:33.694220 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:33.694222 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:33.694300 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:33.694605 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:33.694608 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:33.694610 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:33.694782 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:33.694887 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:33.695139 (+ 252us) server_negotiation.cc:299] Negotiation successful
0929 10:33:33.695257 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":115,"threads_started":1}
I20260929 10:33:33.696147 22524 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.82.124" port: 41423 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 10:33:33 dist-test-slave-vg89 krb5kdc[21759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678013, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.82.124@KRBTEST.COM for oryx/127.15.82.124@KRBTEST.COM
I20260929 10:33:33.703413 22527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:33.696605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.124:41423 (local address 127.0.0.1:38082)
0929 10:33:33.696740 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:33.696756 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:33.696833 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:33.697027 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:33.697030 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:33.697186 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:33.697241 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:33.697246 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.697935 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.697938 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:33.698496 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:33.698504 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.698930 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.698933 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:33.699005 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:33.699458 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:33.699469 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:33.700621 (+ 1152us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:33.702203 (+ 1582us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:33.702205 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:33.702206 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:33.702371 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:33.702667 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:33.702670 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:33.702671 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:33.702740 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:33.703045 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:33.703047 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:33.703113 (+ 66us) client_negotiation.cc:769] Sending connection context
0929 10:33:33.703189 (+ 76us) client_negotiation.cc:240] Negotiation successful
0929 10:33:33.703276 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260929 10:33:33.703444 22526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:33.696640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38082 (local address 127.15.82.124:41423)
0929 10:33:33.696741 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:33.696746 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:33.696798 (+ 52us) server_negotiation.cc:407] Connection header received
0929 10:33:33.696898 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:33.696903 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:33.696941 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:33.697018 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:33.697343 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.697828 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.698598 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.698829 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.698935 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:33.700744 (+ 1809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:33.700746 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:33.700748 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:33.700768 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:33.702113 (+ 1345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:33.702515 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:33.702517 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:33.702519 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:33.702587 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:33.702815 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:33.702817 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:33.702818 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:33.702969 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:33.703041 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:33.703245 (+ 204us) server_negotiation.cc:299] Negotiation successful
0929 10:33:33.703330 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260929 10:33:33.706908 22524 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9df9b9ceadd74c56bc5c0242855d0777: Bootstrap starting.
I20260929 10:33:33.709261 22524 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9df9b9ceadd74c56bc5c0242855d0777: Neither blocks nor log segments found. Creating new log.
I20260929 10:33:33.710000 22524 log.cc:824] T 00000000000000000000000000000000 P 9df9b9ceadd74c56bc5c0242855d0777: Log is configured to *not* fsync() on all Append() calls
I20260929 10:33:33.712096 22524 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9df9b9ceadd74c56bc5c0242855d0777: No bootstrap required, opened a new log
I20260929 10:33:33.715091 22524 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9df9b9ceadd74c56bc5c0242855d0777 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b919a4b6dcc4702b515509aa1dd2164" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 42245 } } peers { permanent_uuid: "ce1288d49e714ca69abeca37c76aebae" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 41741 } } peers { permanent_uuid: "9df9b9ceadd74c56bc5c0242855d0777" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 41423 } }
I20260929 10:33:33.715320 22524 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9df9b9ceadd74c56bc5c0242855d0777 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:33:33.715399 22524 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9df9b9ceadd74c56bc5c0242855d0777 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9df9b9ceadd74c56bc5c0242855d0777, State: Initialized, Role: FOLLOWER
I20260929 10:33:33.715849 22524 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9df9b9ceadd74c56bc5c0242855d0777 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b919a4b6dcc4702b515509aa1dd2164" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 42245 } } peers { permanent_uuid: "ce1288d49e714ca69abeca37c76aebae" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 41741 } } peers { permanent_uuid: "9df9b9ceadd74c56bc5c0242855d0777" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 41423 } }
I20260929 10:33:33.716387 22530 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9df9b9ceadd74c56bc5c0242855d0777 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b919a4b6dcc4702b515509aa1dd2164" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 42245 } } peers { permanent_uuid: "ce1288d49e714ca69abeca37c76aebae" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 41741 } } peers { permanent_uuid: "9df9b9ceadd74c56bc5c0242855d0777" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 41423 } } }
I20260929 10:33:33.716534 22530 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9df9b9ceadd74c56bc5c0242855d0777 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 10:33:33.717319 22524 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9df9b9ceadd74c56bc5c0242855d0777 [sys.catalog]: configured and running, proceeding with master startup.
W20260929 10:33:33.722009 22541 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9df9b9ceadd74c56bc5c0242855d0777: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 10:33:33.722093 22541 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 10:33:33.725304 22378 master_runner.cc:428] Detected that this master 9df9b9ceadd74c56bc5c0242855d0777 is joining an existing cluster
I20260929 10:33:33.725380 22378 master_runner.cc:432] Initiating AddMaster RPC to add 127.15.82.124:41423
I20260929 10:33:33.736614 22549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:33.727414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:42245 (local address 127.0.0.1:37072)
0929 10:33:33.728121 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:33.728138 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:33.728252 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:33.728596 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:33.728599 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:33.728867 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:33.729150 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:33.729161 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.730285 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.730288 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:33.731352 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:33.731361 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.732060 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.732063 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:33.732145 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:33.732713 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:33.732723 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:33.733066 (+ 343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:33.735118 (+ 2052us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:33.735121 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:33.735122 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:33.735458 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:33.735729 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:33.735731 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:33.735733 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:33.735804 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:33.736225 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:33.736227 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:33.736298 (+ 71us) client_negotiation.cc:769] Sending connection context
0929 10:33:33.736374 (+ 76us) client_negotiation.cc:240] Negotiation successful
0929 10:33:33.736471 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":631,"mutex_wait_us":51,"spinlock_wait_cycles":2816,"thread_start_us":83,"threads_started":1}
I20260929 10:33:33.736727 21843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:33.727531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37072 (local address 127.15.82.126:42245)
0929 10:33:33.727672 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:33.727679 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:33.728213 (+ 534us) server_negotiation.cc:407] Connection header received
0929 10:33:33.728333 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:33.728336 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:33.728387 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:33.728475 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:33.729290 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.730149 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.731486 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.731838 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.731904 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:33.733215 (+ 1311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:33.733219 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:33.733222 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:33.733249 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:33.735020 (+ 1771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:33.735542 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:33.735545 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:33.735547 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:33.735639 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:33.735889 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:33.735892 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:33.735894 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:33.736107 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:33.736241 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:33.736453 (+ 212us) server_negotiation.cc:299] Negotiation successful
0929 10:33:33.736570 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260929 10:33:33.739569 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 21866
Sep 29 10:33:33 dist-test-slave-vg89 krb5kdc[21759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678012, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.82.126@KRBTEST.COM for oryx/127.15.82.124@KRBTEST.COM
I20260929 10:33:33.748755 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.65:45833
--local_ip_for_outbound_sockets=127.15.82.65
--tserver_master_addrs=127.15.82.126:42245
--webserver_port=35231
--webserver_interface=127.15.82.65
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/krb5kdc/oryx_127.15.82.65.keytab
--principal=oryx/127.15.82.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:40517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.15.82.126:42245,127.15.82.125:41741
--tserver_master_addrs=127.15.82.126:42245,127.15.82.125:41741,127.15.82.124:41423 with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 10:33:33.751087 22547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:33.727832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.125:41741 (local address 127.0.0.1:42284)
0929 10:33:33.728590 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:33.728603 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:33.728724 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:33.729019 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:33.729022 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:33.729268 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:33.729418 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:33.729426 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.730549 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.730552 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:33.731513 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:33.731524 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.732260 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.732263 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:33.732361 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:33.746895 (+ 14534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:33.746911 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:33.747347 (+ 436us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:33.749321 (+ 1974us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:33.749325 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:33.749327 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:33.749698 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:33.749993 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:33.749996 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:33.749998 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:33.750077 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:33.750552 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:33.750555 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:33.750647 (+ 92us) client_negotiation.cc:769] Sending connection context
0929 10:33:33.750748 (+ 101us) client_negotiation.cc:240] Negotiation successful
0929 10:33:33.750890 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":700,"mutex_wait_us":27,"thread_start_us":82,"threads_started":1}
I20260929 10:33:33.751593 22528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:33.727892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42284 (local address 127.15.82.125:41741)
0929 10:33:33.728008 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:33.728013 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:33.728667 (+ 654us) server_negotiation.cc:407] Connection header received
0929 10:33:33.728843 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:33.728848 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:33.728900 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:33.728986 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:33.729559 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.730376 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.731651 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.732012 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.732091 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:33.747526 (+ 15435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:33.747531 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:33.747534 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:33.747565 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:33.749190 (+ 1625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:33.749812 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:33.749815 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:33.749817 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:33.749899 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:33.750179 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:33.750183 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:33.750184 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:33.750421 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:33.750509 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:33.751307 (+ 798us) server_negotiation.cc:299] Negotiation successful
0929 10:33:33.751438 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260929 10:33:33.752112 22361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:33.740411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.124:41423 (local address 127.0.0.1:38094)
0929 10:33:33.740573 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:33.740590 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:33.740679 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:33.741403 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:33.741409 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:33.741621 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:33.741763 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:33.741774 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.742852 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.742856 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:33.743724 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:33.743734 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.744357 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.744364 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:33.744499 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:33.745022 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:33.745042 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:33.746796 (+ 1754us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:33.749905 (+ 3109us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:33.749911 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:33.749916 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:33.750180 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:33.750565 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:33.750570 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:33.750575 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:33.750686 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:33.751502 (+ 816us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:33.751508 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:33.751618 (+ 110us) client_negotiation.cc:769] Sending connection context
0929 10:33:33.751729 (+ 111us) client_negotiation.cc:240] Negotiation successful
0929 10:33:33.751902 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260929 10:33:33.753618 22550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:33.740823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38094 (local address 127.15.82.124:41423)
0929 10:33:33.741111 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:33.741119 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:33.741132 (+ 13us) server_negotiation.cc:407] Connection header received
0929 10:33:33.741203 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:33.741206 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:33.741264 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:33.741354 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:33.741897 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.742724 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.743868 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.744232 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.744326 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:33.746977 (+ 2651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:33.746984 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:33.746990 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:33.747021 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:33.749781 (+ 2760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:33.750313 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:33.750319 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:33.750321 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:33.750450 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:33.750814 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:33.750818 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:33.750823 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:33.751036 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:33.753186 (+ 2150us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:33.753264 (+ 78us) server_negotiation.cc:299] Negotiation successful
0929 10:33:33.753390 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":221,"spinlock_wait_cycles":384,"thread_start_us":125,"threads_started":1}
I20260929 10:33:33.754773 22548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:33.727669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.124:41423 (local address 127.0.0.1:38090)
0929 10:33:33.728237 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:33.728253 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:33.728373 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:33.728678 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:33.728682 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:33.729234 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:33.729380 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:33.729387 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.730291 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.730293 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:33.731701 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:33.731710 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.732366 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.732372 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:33.732474 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:33.750870 (+ 18396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:33.750884 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:33.751311 (+ 427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:33.753338 (+ 2027us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:33.753341 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:33.753343 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:33.753580 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:33.753845 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:33.753848 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:33.753850 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:33.753917 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:33.754282 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:33.754285 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:33.754353 (+ 68us) client_negotiation.cc:769] Sending connection context
0929 10:33:33.754481 (+ 128us) client_negotiation.cc:240] Negotiation successful
0929 10:33:33.754605 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"mutex_wait_us":51,"thread_start_us":104,"threads_started":1}
I20260929 10:33:33.754856 21799 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.15.82.124:41423: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "9df9b9ceadd74c56bc5c0242855d0777"
member_type: NON_VOTER
last_known_addr {
host: "127.15.82.124"
port: 41423
}
attrs {
promote: true
}
}
dest_uuid: "2b919a4b6dcc4702b515509aa1dd2164"
cas_config_opid_index: 5
I20260929 10:33:33.755239 22526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:33.727987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38090 (local address 127.15.82.124:41423)
0929 10:33:33.728085 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:33.728090 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:33.728318 (+ 228us) server_negotiation.cc:407] Connection header received
0929 10:33:33.728468 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:33.728471 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:33.728506 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:33.728571 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:33.729479 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.730185 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.731804 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.732090 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.732186 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:33.751436 (+ 19250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:33.751439 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:33.751443 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:33.751468 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:33.753234 (+ 1766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:33.753688 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:33.753691 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:33.753693 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:33.753763 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:33.753994 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:33.753997 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:33.753998 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:33.754173 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:33.754252 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:33.754993 (+ 741us) server_negotiation.cc:299] Negotiation successful
0929 10:33:33.755091 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":42,"server-negotiator.queue_time_us":36,"spinlock_wait_cycles":1280}
I20260929 10:33:33.755555 21799 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2b919a4b6dcc4702b515509aa1dd2164 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "2b919a4b6dcc4702b515509aa1dd2164" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 42245 } } peers { permanent_uuid: "ce1288d49e714ca69abeca37c76aebae" member_type: NON_VOTER last_known_addr { host: "127.15.82.125" port: 41741 } attrs { promote: true } } peers { permanent_uuid: "9df9b9ceadd74c56bc5c0242855d0777" member_type: NON_VOTER last_known_addr { host: "127.15.82.124" port: 41423 } attrs { promote: true } }
I20260929 10:33:33.756916 22365 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2b919a4b6dcc4702b515509aa1dd2164 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 9df9b9ceadd74c56bc5c0242855d0777 (127.15.82.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "2b919a4b6dcc4702b515509aa1dd2164" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 42245 } } peers { permanent_uuid: "ce1288d49e714ca69abeca37c76aebae" member_type: NON_VOTER last_known_addr { host: "127.15.82.125" port: 41741 } attrs { promote: true } } peers { permanent_uuid: "9df9b9ceadd74c56bc5c0242855d0777" member_type: NON_VOTER last_known_addr { host: "127.15.82.124" port: 41423 } attrs { promote: true } } }
I20260929 10:33:33.758427 22365 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.15.82.124:41423
I20260929 10:33:33.758608 22365 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b919a4b6dcc4702b515509aa1dd2164 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "2b919a4b6dcc4702b515509aa1dd2164" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "2b919a4b6dcc4702b515509aa1dd2164" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 42245 } } peers { permanent_uuid: "ce1288d49e714ca69abeca37c76aebae" member_type: NON_VOTER last_known_addr { host: "127.15.82.125" port: 41741 } attrs { promote: true } } peers { permanent_uuid: "9df9b9ceadd74c56bc5c0242855d0777" member_type: NON_VOTER last_known_addr { host: "127.15.82.124" port: 41423 } attrs { promote: true } } }
I20260929 10:33:33.758698 22365 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b919a4b6dcc4702b515509aa1dd2164 [sys.catalog]: This master's current role is: LEADER
I20260929 10:33:33.759267 22378 master.cc:537] Master@127.15.82.124:41423 shutting down...
I20260929 10:33:33.759773 22498 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9df9b9ceadd74c56bc5c0242855d0777 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:33:33.760917 22498 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9df9b9ceadd74c56bc5c0242855d0777 [term 1 FOLLOWER]: Refusing update from remote peer 2b919a4b6dcc4702b515509aa1dd2164: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260929 10:33:33.761574 22364 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2b919a4b6dcc4702b515509aa1dd2164 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9df9b9ceadd74c56bc5c0242855d0777" member_type: NON_VOTER last_known_addr { host: "127.15.82.124" port: 41423 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260929 10:33:33.763864 21790 consensus_peers.cc:487] T 00000000000000000000000000000000 P 2b919a4b6dcc4702b515509aa1dd2164 -> Peer 9df9b9ceadd74c56bc5c0242855d0777 (127.15.82.124:41423): Couldn't send request to peer 9df9b9ceadd74c56bc5c0242855d0777. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260929 10:33:33.764448 22528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:33.757389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42292 (local address 127.15.82.125:41741)
0929 10:33:33.758470 (+ 1081us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:33.758476 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:33.758493 (+ 17us) server_negotiation.cc:407] Connection header received
0929 10:33:33.758543 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:33.758549 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:33.758617 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:33.758737 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:33:33.759908 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.760797 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.762562 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.763496 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.763621 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:33.763824 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:33.764215 (+ 391us) server_negotiation.cc:299] Negotiation successful
0929 10:33:33.764303 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":954}
I20260929 10:33:33.764456 22361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:33.757037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.125:41741 (local address 127.0.0.1:42292)
0929 10:33:33.758157 (+ 1120us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:33.758174 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:33.758261 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:33.758780 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:33.758784 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:33.758805 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:33:33.758889 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:33.758894 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.760936 (+ 2042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.760940 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:33.762188 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:33.762203 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.763896 (+ 1693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.763902 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:33.764029 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:33.764090 (+ 61us) client_negotiation.cc:769] Sending connection context
0929 10:33:33.764185 (+ 95us) client_negotiation.cc:240] Negotiation successful
0929 10:33:33.764280 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1025}
I20260929 10:33:33.765157 22416 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ce1288d49e714ca69abeca37c76aebae [term 1 LEARNER]: Refusing update from remote peer 2b919a4b6dcc4702b515509aa1dd2164: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260929 10:33:33.765771 22364 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2b919a4b6dcc4702b515509aa1dd2164 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce1288d49e714ca69abeca37c76aebae" member_type: NON_VOTER last_known_addr { host: "127.15.82.125" port: 41741 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260929 10:33:33.767995 22416 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ce1288d49e714ca69abeca37c76aebae [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 9df9b9ceadd74c56bc5c0242855d0777 (127.15.82.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "2b919a4b6dcc4702b515509aa1dd2164" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 42245 } } peers { permanent_uuid: "ce1288d49e714ca69abeca37c76aebae" member_type: NON_VOTER last_known_addr { host: "127.15.82.125" port: 41741 } attrs { promote: true } } peers { permanent_uuid: "9df9b9ceadd74c56bc5c0242855d0777" member_type: NON_VOTER last_known_addr { host: "127.15.82.124" port: 41423 } attrs { promote: true } } }
I20260929 10:33:33.770278 22445 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce1288d49e714ca69abeca37c76aebae [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b919a4b6dcc4702b515509aa1dd2164. Latest consensus state: current_term: 1 leader_uuid: "2b919a4b6dcc4702b515509aa1dd2164" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "2b919a4b6dcc4702b515509aa1dd2164" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 42245 } } peers { permanent_uuid: "ce1288d49e714ca69abeca37c76aebae" member_type: NON_VOTER last_known_addr { host: "127.15.82.125" port: 41741 } attrs { promote: true } } peers { permanent_uuid: "9df9b9ceadd74c56bc5c0242855d0777" member_type: NON_VOTER last_known_addr { host: "127.15.82.124" port: 41423 } attrs { promote: true } } }
I20260929 10:33:33.770419 22445 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce1288d49e714ca69abeca37c76aebae [sys.catalog]: This master's current role is: LEARNER
I20260929 10:33:33.770507 22445 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce1288d49e714ca69abeca37c76aebae [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2b919a4b6dcc4702b515509aa1dd2164" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "2b919a4b6dcc4702b515509aa1dd2164" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 42245 } } peers { permanent_uuid: "ce1288d49e714ca69abeca37c76aebae" member_type: NON_VOTER last_known_addr { host: "127.15.82.125" port: 41741 } attrs { promote: true } } peers { permanent_uuid: "9df9b9ceadd74c56bc5c0242855d0777" member_type: NON_VOTER last_known_addr { host: "127.15.82.124" port: 41423 } attrs { promote: true } } }
I20260929 10:33:33.770588 22445 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce1288d49e714ca69abeca37c76aebae [sys.catalog]: This master's current role is: LEARNER
I20260929 10:33:33.771390 22366 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 2b919a4b6dcc4702b515509aa1dd2164: attempting to promote NON_VOTER ce1288d49e714ca69abeca37c76aebae to VOTER
I20260929 10:33:33.771404 22378 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 9df9b9ceadd74c56bc5c0242855d0777 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:33:33.771695 22378 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 9df9b9ceadd74c56bc5c0242855d0777 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:33:33.771785 22378 tablet_replica.cc:330] T 00000000000000000000000000000000 P 9df9b9ceadd74c56bc5c0242855d0777: stopping tablet replica
I20260929 10:33:33.771874 22366 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2b919a4b6dcc4702b515509aa1dd2164 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2b919a4b6dcc4702b515509aa1dd2164" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 42245 } } peers { permanent_uuid: "ce1288d49e714ca69abeca37c76aebae" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 41741 } attrs { promote: false } } peers { permanent_uuid: "9df9b9ceadd74c56bc5c0242855d0777" member_type: NON_VOTER last_known_addr { host: "127.15.82.124" port: 41423 } attrs { promote: true } }
I20260929 10:33:33.772993 22416 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ce1288d49e714ca69abeca37c76aebae [term 1 LEARNER]: Refusing update from remote peer 2b919a4b6dcc4702b515509aa1dd2164: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260929 10:33:33.773339 22365 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2b919a4b6dcc4702b515509aa1dd2164 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce1288d49e714ca69abeca37c76aebae" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 41741 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260929 10:33:33.774505 21790 consensus_peers.cc:487] T 00000000000000000000000000000000 P 2b919a4b6dcc4702b515509aa1dd2164 -> Peer 9df9b9ceadd74c56bc5c0242855d0777 (127.15.82.124:41423): Couldn't send request to peer 9df9b9ceadd74c56bc5c0242855d0777. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260929 10:33:33.775199 22365 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2b919a4b6dcc4702b515509aa1dd2164 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, ce1288d49e714ca69abeca37c76aebae (127.15.82.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2b919a4b6dcc4702b515509aa1dd2164" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 42245 } } peers { permanent_uuid: "ce1288d49e714ca69abeca37c76aebae" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 41741 } attrs { promote: false } } peers { permanent_uuid: "9df9b9ceadd74c56bc5c0242855d0777" member_type: NON_VOTER last_known_addr { host: "127.15.82.124" port: 41423 } attrs { promote: true } } }
I20260929 10:33:33.776619 22365 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b919a4b6dcc4702b515509aa1dd2164 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "2b919a4b6dcc4702b515509aa1dd2164" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2b919a4b6dcc4702b515509aa1dd2164" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 42245 } } peers { permanent_uuid: "ce1288d49e714ca69abeca37c76aebae" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 41741 } attrs { promote: false } } peers { permanent_uuid: "9df9b9ceadd74c56bc5c0242855d0777" member_type: NON_VOTER last_known_addr { host: "127.15.82.124" port: 41423 } attrs { promote: true } } }
I20260929 10:33:33.776726 22365 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b919a4b6dcc4702b515509aa1dd2164 [sys.catalog]: This master's current role is: LEADER
I20260929 10:33:33.776870 22365 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b919a4b6dcc4702b515509aa1dd2164 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2b919a4b6dcc4702b515509aa1dd2164" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2b919a4b6dcc4702b515509aa1dd2164" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 42245 } } peers { permanent_uuid: "ce1288d49e714ca69abeca37c76aebae" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 41741 } attrs { promote: false } } peers { permanent_uuid: "9df9b9ceadd74c56bc5c0242855d0777" member_type: NON_VOTER last_known_addr { host: "127.15.82.124" port: 41423 } attrs { promote: true } } }
I20260929 10:33:33.776945 22365 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b919a4b6dcc4702b515509aa1dd2164 [sys.catalog]: This master's current role is: LEADER
I20260929 10:33:33.777704 22416 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ce1288d49e714ca69abeca37c76aebae [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, ce1288d49e714ca69abeca37c76aebae (127.15.82.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2b919a4b6dcc4702b515509aa1dd2164" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 42245 } } peers { permanent_uuid: "ce1288d49e714ca69abeca37c76aebae" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 41741 } attrs { promote: false } } peers { permanent_uuid: "9df9b9ceadd74c56bc5c0242855d0777" member_type: NON_VOTER last_known_addr { host: "127.15.82.124" port: 41423 } attrs { promote: true } } }
I20260929 10:33:33.779395 22553 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce1288d49e714ca69abeca37c76aebae [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2b919a4b6dcc4702b515509aa1dd2164" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2b919a4b6dcc4702b515509aa1dd2164" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 42245 } } peers { permanent_uuid: "ce1288d49e714ca69abeca37c76aebae" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 41741 } attrs { promote: false } } peers { permanent_uuid: "9df9b9ceadd74c56bc5c0242855d0777" member_type: NON_VOTER last_known_addr { host: "127.15.82.124" port: 41423 } attrs { promote: true } } }
I20260929 10:33:33.779518 22553 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce1288d49e714ca69abeca37c76aebae [sys.catalog]: This master's current role is: FOLLOWER
I20260929 10:33:33.788030 22378 master.cc:552] Master@127.15.82.124:41423 shutdown complete.
I20260929 10:33:33.788098 22378 master_runner.cc:305] Clearing existing system tablet
I20260929 10:33:33.788342 22378 fs_manager.cc:359] Metadata directory not provided
I20260929 10:33:33.788400 22378 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:33:33.789644 22378 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:33:33.790541 22555 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:33.790781 22378 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:33:33.790907 22378 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:33:33.790940 22378 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/master-2/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/master-2/wal
uuid: "9df9b9ceadd74c56bc5c0242855d0777"
format_stamp: "Formatted at 2026-09-29 10:33:33 on dist-test-slave-vg89"
server_key: "f8399c33eac1e58e8bd44adf477a914b"
server_key_iv: "bcd1b025f9186a5fe8386736ed50351d"
server_key_version: "encryptionkey@0"
I20260929 10:33:33.793054 22378 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 9df9b9ceadd74c56bc5c0242855d0777: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:33:33.795557 22378 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 9df9b9ceadd74c56bc5c0242855d0777: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260929 10:33:33.795650 22378 log.cc:1196] T 00000000000000000000000000000000 P 9df9b9ceadd74c56bc5c0242855d0777: Deleting WAL directory at /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260929 10:33:33.795899 22378 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 9df9b9ceadd74c56bc5c0242855d0777: Deleting consensus metadata
I20260929 10:33:33.796088 22378 master_runner.cc:315] Copying system tablet from 127.15.82.126:42245
I20260929 10:33:33.797260 22378 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 9df9b9ceadd74c56bc5c0242855d0777: tablet copy: Beginning tablet copy session from remote peer at address 127.15.82.126:42245
I20260929 10:33:33.806384 21843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:33.797866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37078 (local address 127.15.82.126:42245)
0929 10:33:33.797995 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:33.797999 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:33.798247 (+ 248us) server_negotiation.cc:407] Connection header received
0929 10:33:33.798354 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:33.798357 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:33.798449 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:33.798539 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:33.799059 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.799923 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.801135 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.801468 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.801583 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:33.802931 (+ 1348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:33.802937 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:33.802941 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:33.802977 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:33.804664 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:33.805135 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:33.805139 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:33.805141 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:33.805229 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:33.805513 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:33.805516 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:33.805517 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:33.805729 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:33.805827 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:33.806109 (+ 282us) server_negotiation.cc:299] Negotiation successful
0929 10:33:33.806226 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260929 10:33:33.806433 22561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:33.797798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:42245 (local address 127.0.0.1:37078)
0929 10:33:33.798125 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:33.798140 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:33.798264 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:33.798571 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:33.798574 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:33.798800 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:33.798927 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:33.798933 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.800043 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.800047 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:33.801007 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:33.801017 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.801590 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.801594 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:33.801698 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:33.802291 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:33.802306 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:33.802781 (+ 475us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:33.804777 (+ 1996us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:33.804780 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:33.804783 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:33.805035 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:33.805324 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:33.805327 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:33.805329 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:33.805420 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:33.805827 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:33.805830 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:33.805929 (+ 99us) client_negotiation.cc:769] Sending connection context
0929 10:33:33.806035 (+ 106us) client_negotiation.cc:240] Negotiation successful
0929 10:33:33.806179 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":89,"threads_started":1}
I20260929 10:33:33.807194 21824 tablet_copy_service.cc:140] P 2b919a4b6dcc4702b515509aa1dd2164: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 9df9b9ceadd74c56bc5c0242855d0777 ({username='oryx', principal='oryx/127.15.82.124@KRBTEST.COM'} at 127.0.0.1:37078)
I20260929 10:33:33.807297 21824 tablet_copy_service.cc:161] P 2b919a4b6dcc4702b515509aa1dd2164: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 9df9b9ceadd74c56bc5c0242855d0777 at {username='oryx', principal='oryx/127.15.82.124@KRBTEST.COM'} at 127.0.0.1:37078: session id = 9df9b9ceadd74c56bc5c0242855d0777-00000000000000000000000000000000
I20260929 10:33:33.808503 21824 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 2b919a4b6dcc4702b515509aa1dd2164: Tablet Copy: opened 0 blocks and 1 log segments
I20260929 10:33:33.809927 22378 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260929 10:33:33.812397 22378 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 9df9b9ceadd74c56bc5c0242855d0777: tablet copy: Starting download of 0 data blocks...
I20260929 10:33:33.812592 22378 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 9df9b9ceadd74c56bc5c0242855d0777: tablet copy: Starting download of 1 WAL segments...
I20260929 10:33:33.814284 22378 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 9df9b9ceadd74c56bc5c0242855d0777: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260929 10:33:33.816169 21824 tablet_copy_service.cc:342] P 2b919a4b6dcc4702b515509aa1dd2164: Request end of tablet copy session 9df9b9ceadd74c56bc5c0242855d0777-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.15.82.124@KRBTEST.COM'} at 127.0.0.1:37078
I20260929 10:33:33.816264 21824 tablet_copy_service.cc:433] P 2b919a4b6dcc4702b515509aa1dd2164: ending tablet copy session 9df9b9ceadd74c56bc5c0242855d0777-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 9df9b9ceadd74c56bc5c0242855d0777
I20260929 10:33:33.824879 22378 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:33:33.826637 22562 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:33.831270 22563 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:33.831440 22565 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:33.833894 22378 server_base.cc:1034] running on GCE node
I20260929 10:33:33.834082 22378 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:33:33.834448 22378 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:33:33.835611 22378 hybrid_clock.cc:648] HybridClock initialized: now 1790678013835591 us; error 41 us; skew 500 ppm
Sep 29 10:33:33 dist-test-slave-vg89 krb5kdc[21759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678013, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.82.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:33:33.837203 22378 init.cc:377] Logged in from keytab as oryx/127.15.82.124@KRBTEST.COM (short username oryx)
I20260929 10:33:33.837661 22378 webserver.cc:460] Webserver started at http://127.15.82.124:34975/ using document root <none> and password file <none>
I20260929 10:33:33.837848 22378 fs_manager.cc:359] Metadata directory not provided
I20260929 10:33:33.837888 22378 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:33:33.839274 22378 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 10:33:33.840340 22572 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:33.840610 22378 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:33:33.840730 22378 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/master-2/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/master-2/wal
uuid: "9df9b9ceadd74c56bc5c0242855d0777"
format_stamp: "Formatted at 2026-09-29 10:33:33 on dist-test-slave-vg89"
server_key: "f8399c33eac1e58e8bd44adf477a914b"
server_key_iv: "bcd1b025f9186a5fe8386736ed50351d"
server_key_version: "encryptionkey@0"
I20260929 10:33:33.840878 22378 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:33:33.854151 22378 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:33:33.856451 22378 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:33:33.862212 22378 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.124:41423
I20260929 10:33:33.862259 22624 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.124:41423 every 8 connection(s)
I20260929 10:33:33.862990 22378 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/master-2/data/info.pb
I20260929 10:33:33.865473 22625 sys_catalog.cc:263] Verifying existing consensus state
I20260929 10:33:33.866726 22625 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9df9b9ceadd74c56bc5c0242855d0777: Bootstrap starting.
I20260929 10:33:33.878712 22625 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9df9b9ceadd74c56bc5c0242855d0777: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 10:33:33.879086 22625 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9df9b9ceadd74c56bc5c0242855d0777: Bootstrap complete.
I20260929 10:33:33.879712 22625 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9df9b9ceadd74c56bc5c0242855d0777 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2b919a4b6dcc4702b515509aa1dd2164" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 42245 } } peers { permanent_uuid: "ce1288d49e714ca69abeca37c76aebae" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 41741 } attrs { promote: false } } peers { permanent_uuid: "9df9b9ceadd74c56bc5c0242855d0777" member_type: NON_VOTER last_known_addr { host: "127.15.82.124" port: 41423 } attrs { promote: true } }
I20260929 10:33:33.879894 22625 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9df9b9ceadd74c56bc5c0242855d0777 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 9df9b9ceadd74c56bc5c0242855d0777, State: Initialized, Role: LEARNER
I20260929 10:33:33.880013 22625 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9df9b9ceadd74c56bc5c0242855d0777 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2b919a4b6dcc4702b515509aa1dd2164" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 42245 } } peers { permanent_uuid: "ce1288d49e714ca69abeca37c76aebae" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 41741 } attrs { promote: false } } peers { permanent_uuid: "9df9b9ceadd74c56bc5c0242855d0777" member_type: NON_VOTER last_known_addr { host: "127.15.82.124" port: 41423 } attrs { promote: true } }
I20260929 10:33:33.880384 22628 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9df9b9ceadd74c56bc5c0242855d0777 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2b919a4b6dcc4702b515509aa1dd2164" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 42245 } } peers { permanent_uuid: "ce1288d49e714ca69abeca37c76aebae" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 41741 } attrs { promote: false } } peers { permanent_uuid: "9df9b9ceadd74c56bc5c0242855d0777" member_type: NON_VOTER last_known_addr { host: "127.15.82.124" port: 41423 } attrs { promote: true } } }
I20260929 10:33:33.880553 22625 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9df9b9ceadd74c56bc5c0242855d0777 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:33:33.880582 22628 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9df9b9ceadd74c56bc5c0242855d0777 [sys.catalog]: This master's current role is: LEARNER
I20260929 10:33:33.884629 22639 catalog_manager.cc:1259] Loaded cluster ID: 1f355f33ab054e5c8cc0fe255d71c834
I20260929 10:33:33.884809 22639 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9df9b9ceadd74c56bc5c0242855d0777: loading cluster ID for follower catalog manager: success
I20260929 10:33:33.887187 22639 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9df9b9ceadd74c56bc5c0242855d0777: acquiring CA information for follower catalog manager: success
I20260929 10:33:33.888983 22639 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9df9b9ceadd74c56bc5c0242855d0777: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260929 10:33:33.895244 22551 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:33:33.895533 22551 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:33:33.895641 22551 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:33:33.899945 22551 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:33:33.900029 22551 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:33:33.900127 22551 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.65
I20260929 10:33:33.907061 22551 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:40517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/krb5kdc/oryx_127.15.82.65.keytab
--principal=oryx/127.15.82.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.65:45833
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.82.65
--webserver_port=35231
--tserver_master_addrs=127.15.82.126:42245,127.15.82.125:41741,127.15.82.124:41423
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22551
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.65
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:33:33.908147 22551 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:33:33.909201 22551 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:33:33.919221 22644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:33.919252 22647 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:33.919322 22551 server_base.cc:1034] running on GCE node
W20260929 10:33:33.919252 22645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:33.919900 22551 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:33:33.920570 22551 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:33:33.921731 22551 hybrid_clock.cc:648] HybridClock initialized: now 1790678013921714 us; error 30 us; skew 500 ppm
I20260929 10:33:33.923100 22129 heartbeater.cc:503] Master 127.15.82.126:42245 was elected leader, sending a full tablet report...
Sep 29 10:33:33 dist-test-slave-vg89 krb5kdc[21759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678013, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.82.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:33:33.924834 22551 init.cc:377] Logged in from keytab as oryx/127.15.82.65@KRBTEST.COM (short username oryx)
I20260929 10:33:33.925693 22551 webserver.cc:460] Webserver started at http://127.15.82.65:35231/ using document root <none> and password file <none>
I20260929 10:33:33.926224 22551 fs_manager.cc:359] Metadata directory not provided
I20260929 10:33:33.926297 22551 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:33:33.930195 22551 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260929 10:33:33.932279 22654 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:33.933542 22551 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260929 10:33:33.933691 22551 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-0/wal
uuid: "068171b1b6514121936f4634de5a8064"
format_stamp: "Formatted at 2026-09-29 10:33:32 on dist-test-slave-vg89"
server_key: "8249a5889aa23be947694a761cd99c0a"
server_key_iv: "2ecebf5ab4709fb59969bfb0b84efb15"
server_key_version: "encryptionkey@0"
I20260929 10:33:33.934260 22551 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:33:33.956276 22551 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:33:33.959599 22551 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:33:33.959841 22551 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:33:33.960559 22551 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:33:33.961481 22551 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:33:33.961555 22551 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:33.961633 22551 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:33:33.961685 22551 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:33.971426 22551 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.65:45833
I20260929 10:33:33.971459 22767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.65:45833 every 8 connection(s)
I20260929 10:33:33.972492 22551 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
I20260929 10:33:33.980897 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 22551
I20260929 10:33:33.981140 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 22000
Sep 29 10:33:33 dist-test-slave-vg89 krb5kdc[21759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678013, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.82.65@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM
Sep 29 10:33:33 dist-test-slave-vg89 krb5kdc[21759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678013, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.82.65@KRBTEST.COM for oryx/127.15.82.124@KRBTEST.COM
Sep 29 10:33:33 dist-test-slave-vg89 krb5kdc[21759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678013, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.82.65@KRBTEST.COM for oryx/127.15.82.125@KRBTEST.COM
I20260929 10:33:33.990703 22528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:33.976329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.65:43243 (local address 127.15.82.125:41741)
0929 10:33:33.976447 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:33.976450 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:33.976585 (+ 135us) server_negotiation.cc:407] Connection header received
0929 10:33:33.977137 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:33.977141 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:33.977201 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:33.977272 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:33.978736 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.979576 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.982090 (+ 2514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.983152 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.983240 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:33.986525 (+ 3285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:33.986532 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:33.986538 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:33.986576 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:33.988374 (+ 1798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:33.989023 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:33.989029 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:33.989034 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:33.989126 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:33.989438 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:33.989443 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:33.989448 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:33.989734 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:33.989819 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:33.990341 (+ 522us) server_negotiation.cc:299] Negotiation successful
0929 10:33:33.990521 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260929 10:33:33.990705 22772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:33.974539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.65:46471 (local address 127.15.82.124:41423)
0929 10:33:33.974822 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:33.974826 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:33.975820 (+ 994us) server_negotiation.cc:407] Connection header received
0929 10:33:33.977169 (+ 1349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:33.977172 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:33.977226 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:33.977288 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:33.979741 (+ 2453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.980405 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.981596 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.981977 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.982070 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:33.985685 (+ 3615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:33.985689 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:33.985692 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:33.985747 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:33.988012 (+ 2265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:33.988730 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:33.988734 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:33.988735 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:33.988835 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:33.989264 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:33.989267 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:33.989268 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:33.989526 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:33.989619 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:33.990349 (+ 730us) server_negotiation.cc:299] Negotiation successful
0929 10:33:33.990520 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":134,"threads_started":1}
I20260929 10:33:33.990703 21843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:33.974450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.65:37713 (local address 127.15.82.126:42245)
0929 10:33:33.974598 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:33.974604 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:33.975819 (+ 1215us) server_negotiation.cc:407] Connection header received
0929 10:33:33.977162 (+ 1343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:33.977166 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:33.977226 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:33.977285 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:33.978784 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.979576 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.980989 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.981385 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.981533 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:33.986304 (+ 4771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:33.986308 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:33.986311 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:33.986345 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:33.988747 (+ 2402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:33.989248 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:33.989252 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:33.989254 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:33.989348 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:33.989642 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:33.989645 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:33.989647 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:33.989868 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:33.989948 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:33.990349 (+ 401us) server_negotiation.cc:299] Negotiation successful
0929 10:33:33.990516 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 10:33:33.991017 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.66:43097
--local_ip_for_outbound_sockets=127.15.82.66
--tserver_master_addrs=127.15.82.126:42245
--webserver_port=43249
--webserver_interface=127.15.82.66
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/krb5kdc/oryx_127.15.82.66.keytab
--principal=oryx/127.15.82.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:40517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.15.82.126:42245,127.15.82.125:41741
--tserver_master_addrs=127.15.82.126:42245,127.15.82.125:41741,127.15.82.124:41423 with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 10:33:33.992570 22775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:33.976203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.125:41741 (local address 127.15.82.65:43243)
0929 10:33:33.976482 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:33.976496 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:33.976919 (+ 423us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:33.977499 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:33.977502 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:33.977928 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:33.978591 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:33.978607 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.980989 (+ 2382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.980992 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:33.981951 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:33.981962 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.983407 (+ 1445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.983410 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:33.983519 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:33.984211 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:33.984230 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:33.986338 (+ 2108us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:33.988577 (+ 2239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:33.988583 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:33.988588 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:33.988905 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:33.989241 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:33.989246 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:33.989251 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:33.989339 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:33.989917 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:33.989923 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:33.990066 (+ 143us) client_negotiation.cc:769] Sending connection context
0929 10:33:33.990876 (+ 810us) client_negotiation.cc:240] Negotiation successful
0929 10:33:33.991330 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"mutex_wait_us":88,"spinlock_wait_cycles":5632,"thread_start_us":110,"threads_started":1}
I20260929 10:33:33.992574 22773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:33.975038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.124:41423 (local address 127.15.82.65:46471)
0929 10:33:33.975638 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:33.975703 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:33.976905 (+ 1202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:33.977469 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:33.977479 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:33.977992 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:33.978603 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:33.978610 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.980551 (+ 1941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.980555 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:33.981415 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:33.981424 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.982236 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.982239 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:33.982327 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:33.983760 (+ 1433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:33.983776 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:33.985498 (+ 1722us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:33.988178 (+ 2680us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:33.988185 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:33.988207 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:33.988592 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:33.988945 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:33.988948 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:33.988949 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:33.989142 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:33.989684 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:33.989692 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:33.990051 (+ 359us) client_negotiation.cc:769] Sending connection context
0929 10:33:33.991002 (+ 951us) client_negotiation.cc:240] Negotiation successful
0929 10:33:33.991440 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"mutex_wait_us":255,"spinlock_wait_cycles":27264,"thread_start_us":140,"threads_started":1}
I20260929 10:33:33.993033 22774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:33.975038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:42245 (local address 127.15.82.65:37713)
0929 10:33:33.975638 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:33.975703 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:33.976905 (+ 1202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:33.977469 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:33.977479 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:33.978017 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:33.978601 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:33.978610 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.979726 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.979729 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:33.980856 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:33.980867 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:33.981801 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:33.981808 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:33.982090 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:33.982855 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:33.982880 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:33.986105 (+ 3225us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:33.988884 (+ 2779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:33.988891 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:33.988893 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:33.989121 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:33.989458 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:33.989461 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:33.989463 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:33.989541 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:33.990004 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:33.990006 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:33.990072 (+ 66us) client_negotiation.cc:769] Sending connection context
0929 10:33:33.991219 (+ 1147us) client_negotiation.cc:240] Negotiation successful
0929 10:33:33.991496 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":428,"mutex_wait_us":267,"spinlock_wait_cycles":94592,"thread_start_us":135,"threads_started":1}
I20260929 10:33:33.994122 22769 heartbeater.cc:347] Connected to a master server at 127.15.82.126:42245
I20260929 10:33:33.994127 22768 heartbeater.cc:347] Connected to a master server at 127.15.82.124:41423
I20260929 10:33:33.994386 22768 heartbeater.cc:464] Registering TS with master...
I20260929 10:33:33.994187 22770 heartbeater.cc:347] Connected to a master server at 127.15.82.125:41741
I20260929 10:33:33.994436 22769 heartbeater.cc:464] Registering TS with master...
I20260929 10:33:33.994522 22770 heartbeater.cc:464] Registering TS with master...
I20260929 10:33:33.995077 22770 heartbeater.cc:511] Master 127.15.82.125:41741 requested a full tablet report, sending...
I20260929 10:33:33.995077 22768 heartbeater.cc:511] Master 127.15.82.124:41423 requested a full tablet report, sending...
I20260929 10:33:33.995132 22769 heartbeater.cc:511] Master 127.15.82.126:42245 requested a full tablet report, sending...
I20260929 10:33:33.996528 21799 ts_manager.cc:194] Re-registered known tserver with Master: 068171b1b6514121936f4634de5a8064 (127.15.82.65:45833)
I20260929 10:33:33.996770 22406 ts_manager.cc:194] Registered new tserver with Master: 068171b1b6514121936f4634de5a8064 (127.15.82.65:45833)
I20260929 10:33:33.996783 22586 ts_manager.cc:194] Registered new tserver with Master: 068171b1b6514121936f4634de5a8064 (127.15.82.65:45833)
I20260929 10:33:33.997319 21799 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.82.65@KRBTEST.COM'} at 127.15.82.65:37713
W20260929 10:33:34.110337 22776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:33:34.110611 22776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:33:34.110656 22776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:33:34.114665 22776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:33:34.114720 22776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:33:34.114777 22776 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.66
I20260929 10:33:34.119984 22776 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:40517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/krb5kdc/oryx_127.15.82.66.keytab
--principal=oryx/127.15.82.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.66:43097
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.82.66
--webserver_port=43249
--tserver_master_addrs=127.15.82.126:42245,127.15.82.125:41741,127.15.82.124:41423
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22776
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.66
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:33:34.121043 22776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:33:34.122047 22776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:33:34.131922 22784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:34.131930 22781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:34.131940 22782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:34.132290 22776 server_base.cc:1034] running on GCE node
I20260929 10:33:34.132704 22776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:33:34.133363 22776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:33:34.134542 22776 hybrid_clock.cc:648] HybridClock initialized: now 1790678014134523 us; error 55 us; skew 500 ppm
Sep 29 10:33:34 dist-test-slave-vg89 krb5kdc[21759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678014, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.82.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:33:34.137315 22776 init.cc:377] Logged in from keytab as oryx/127.15.82.66@KRBTEST.COM (short username oryx)
I20260929 10:33:34.138295 22776 webserver.cc:460] Webserver started at http://127.15.82.66:43249/ using document root <none> and password file <none>
I20260929 10:33:34.138942 22776 fs_manager.cc:359] Metadata directory not provided
I20260929 10:33:34.139021 22776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:33:34.143208 22776 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260929 10:33:34.145277 22791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:34.146591 22776 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260929 10:33:34.146714 22776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-1/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-1/wal
uuid: "9886d2dce2354767a0ee1823c1021808"
format_stamp: "Formatted at 2026-09-29 10:33:32 on dist-test-slave-vg89"
server_key: "4490963b4c67eb9bbe5f694a9151397c"
server_key_iv: "f9efb776d4d2197828ca21e428fe2955"
server_key_version: "encryptionkey@0"
I20260929 10:33:34.147296 22776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:33:34.149487 22263 heartbeater.cc:503] Master 127.15.82.126:42245 was elected leader, sending a full tablet report...
I20260929 10:33:34.164448 22776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:33:34.167445 22776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:33:34.167655 22776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:33:34.168264 22776 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:33:34.169137 22776 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:33:34.169183 22776 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:34.169250 22776 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:33:34.169279 22776 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:34.178025 22776 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.66:43097
I20260929 10:33:34.178054 22904 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.66:43097 every 8 connection(s)
I20260929 10:33:34.179085 22776 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
I20260929 10:33:34.180120 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 22776
I20260929 10:33:34.180315 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 22134
I20260929 10:33:34.189085 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.67:43005
--local_ip_for_outbound_sockets=127.15.82.67
--tserver_master_addrs=127.15.82.126:42245
--webserver_port=34921
--webserver_interface=127.15.82.67
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/krb5kdc/oryx_127.15.82.67.keytab
--principal=oryx/127.15.82.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:40517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.15.82.126:42245,127.15.82.125:41741
--tserver_master_addrs=127.15.82.126:42245,127.15.82.125:41741,127.15.82.124:41423 with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 10:33:34 dist-test-slave-vg89 krb5kdc[21759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678014, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.82.66@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM
Sep 29 10:33:34 dist-test-slave-vg89 krb5kdc[21759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678014, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.82.66@KRBTEST.COM for oryx/127.15.82.124@KRBTEST.COM
Sep 29 10:33:34 dist-test-slave-vg89 krb5kdc[21759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678014, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.82.66@KRBTEST.COM for oryx/127.15.82.125@KRBTEST.COM
I20260929 10:33:34.198088 22772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:34.182978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:44731 (local address 127.15.82.124:41423)
0929 10:33:34.183114 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:34.183121 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:34.183138 (+ 17us) server_negotiation.cc:407] Connection header received
0929 10:33:34.184283 (+ 1145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:34.184287 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:34.184338 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:34.184391 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:34.186466 (+ 2075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:34.187295 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:34.189521 (+ 2226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:34.189863 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:34.189940 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:34.194321 (+ 4381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:34.194330 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:34.194334 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:34.194367 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:34.196326 (+ 1959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:34.196796 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:34.196799 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:34.196801 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:34.196874 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:34.197165 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:34.197169 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:34.197171 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:34.197367 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:34.197442 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:34.197790 (+ 348us) server_negotiation.cc:299] Negotiation successful
0929 10:33:34.197917 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260929 10:33:34.198088 21843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:34.183808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:49549 (local address 127.15.82.126:42245)
0929 10:33:34.183944 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:34.183951 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:34.184252 (+ 301us) server_negotiation.cc:407] Connection header received
0929 10:33:34.184607 (+ 355us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:34.184614 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:34.184687 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:34.184747 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:34.186467 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:34.187394 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:34.189393 (+ 1999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:34.189801 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:34.189891 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:34.193348 (+ 3457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:34.193354 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:34.193357 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:34.193395 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:34.195630 (+ 2235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:34.196292 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:34.196295 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:34.196297 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:34.196385 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:34.196783 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:34.196786 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:34.196788 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:34.197030 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:34.197103 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:34.197790 (+ 687us) server_negotiation.cc:299] Negotiation successful
0929 10:33:34.197916 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260929 10:33:34.199445 22528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:34.181812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:52903 (local address 127.15.82.125:41741)
0929 10:33:34.182566 (+ 754us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:34.182573 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:34.184034 (+ 1461us) server_negotiation.cc:407] Connection header received
0929 10:33:34.184272 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:34.184276 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:34.184328 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:34.184385 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:34.186462 (+ 2077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:34.187294 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:34.189395 (+ 2101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:34.189728 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:34.189801 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:34.195072 (+ 5271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:34.195077 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:34.195083 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:34.195119 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:34.196976 (+ 1857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:34.197563 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:34.197569 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:34.197572 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:34.197695 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:34.198070 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:34.198078 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:34.198080 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:34.198353 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:34.198618 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:34.199077 (+ 459us) server_negotiation.cc:299] Negotiation successful
0929 10:33:34.199227 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":650}
I20260929 10:33:34.199865 22909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:34.181543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.124:41423 (local address 127.15.82.66:44731)
0929 10:33:34.182898 (+ 1355us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:34.182952 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:34.184042 (+ 1090us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:34.184794 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:34.184803 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:34.185634 (+ 831us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:34.186267 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:34.186287 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:34.187481 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:34.187485 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:34.189407 (+ 1922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:34.189415 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:34.190075 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:34.190078 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:34.190241 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:34.191092 (+ 851us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:34.191108 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:34.194149 (+ 3041us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:34.196440 (+ 2291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:34.196446 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:34.196451 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:34.196691 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:34.196979 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:34.196982 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:34.196985 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:34.197062 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:34.197498 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:34.197501 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:34.197567 (+ 66us) client_negotiation.cc:769] Sending connection context
0929 10:33:34.198254 (+ 687us) client_negotiation.cc:240] Negotiation successful
0929 10:33:34.198684 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1114,"mutex_wait_us":251,"thread_start_us":672,"threads_started":1}
I20260929 10:33:34.199865 22911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:34.183677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:42245 (local address 127.15.82.66:49549)
0929 10:33:34.184150 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:34.184165 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:34.184327 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:34.184921 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:34.184927 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:34.185585 (+ 658us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:34.186279 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:34.186294 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:34.187552 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:34.187557 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:34.189227 (+ 1670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:34.189240 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:34.190078 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:34.190084 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:34.190300 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:34.191032 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:34.191059 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:34.193144 (+ 2085us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:34.195764 (+ 2620us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:34.195772 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:34.195793 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:34.196167 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:34.196492 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:34.196498 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:34.196500 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:34.196666 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:34.197192 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:34.197201 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:34.197513 (+ 312us) client_negotiation.cc:769] Sending connection context
0929 10:33:34.198699 (+ 1186us) client_negotiation.cc:240] Negotiation successful
0929 10:33:34.198859 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":384,"mutex_wait_us":549,"spinlock_wait_cycles":11520,"thread_start_us":76,"threads_started":1}
I20260929 10:33:34.200975 22910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:34.181543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.125:41741 (local address 127.15.82.66:52903)
0929 10:33:34.183948 (+ 2405us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:34.183961 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:34.184097 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:34.184794 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:34.184803 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:34.185446 (+ 643us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:34.186267 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:34.186287 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:34.187481 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:34.187485 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:34.189224 (+ 1739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:34.189238 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:34.190004 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:34.190010 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:34.190216 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:34.191196 (+ 980us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:34.191212 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:34.194880 (+ 3668us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:34.197130 (+ 2250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:34.197136 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:34.197139 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:34.197433 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:34.197832 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:34.197837 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:34.197841 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:34.197940 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:34.198691 (+ 751us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:34.198697 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:34.198856 (+ 159us) client_negotiation.cc:769] Sending connection context
0929 10:33:34.199678 (+ 822us) client_negotiation.cc:240] Negotiation successful
0929 10:33:34.199856 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2307,"mutex_wait_us":699,"spinlock_wait_cycles":6272,"thread_start_us":159,"threads_started":1}
I20260929 10:33:34.201289 22906 heartbeater.cc:347] Connected to a master server at 127.15.82.126:42245
I20260929 10:33:34.201550 22906 heartbeater.cc:464] Registering TS with master...
I20260929 10:33:34.202173 22906 heartbeater.cc:511] Master 127.15.82.126:42245 requested a full tablet report, sending...
I20260929 10:33:34.202665 22905 heartbeater.cc:347] Connected to a master server at 127.15.82.124:41423
I20260929 10:33:34.202764 22905 heartbeater.cc:464] Registering TS with master...
I20260929 10:33:34.202931 22905 heartbeater.cc:511] Master 127.15.82.124:41423 requested a full tablet report, sending...
I20260929 10:33:34.203141 22907 heartbeater.cc:347] Connected to a master server at 127.15.82.125:41741
I20260929 10:33:34.203209 22907 heartbeater.cc:464] Registering TS with master...
I20260929 10:33:34.203354 22907 heartbeater.cc:511] Master 127.15.82.125:41741 requested a full tablet report, sending...
I20260929 10:33:34.203434 21799 ts_manager.cc:194] Re-registered known tserver with Master: 9886d2dce2354767a0ee1823c1021808 (127.15.82.66:43097)
I20260929 10:33:34.203634 22586 ts_manager.cc:194] Registered new tserver with Master: 9886d2dce2354767a0ee1823c1021808 (127.15.82.66:43097)
I20260929 10:33:34.204043 22406 ts_manager.cc:194] Registered new tserver with Master: 9886d2dce2354767a0ee1823c1021808 (127.15.82.66:43097)
I20260929 10:33:34.204190 21799 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.82.66@KRBTEST.COM'} at 127.15.82.66:49549
I20260929 10:33:34.295300 22913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:34.291096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.124:41423 (local address 127.0.0.1:38102)
0929 10:33:34.291477 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:34.291490 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:34.291594 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:34.291903 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:34.291905 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:34.291920 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:33:34.292021 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:34.292026 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:34.293076 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:34.293079 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:34.293872 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:34.293882 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:34.294849 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:34.294852 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:34.294972 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:34.295014 (+ 42us) client_negotiation.cc:769] Sending connection context
0929 10:33:34.295086 (+ 72us) client_negotiation.cc:240] Negotiation successful
0929 10:33:34.295159 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":96,"threads_started":1}
I20260929 10:33:34.295324 22772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:34.291281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38102 (local address 127.15.82.124:41423)
0929 10:33:34.291428 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:34.291434 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:34.291567 (+ 133us) server_negotiation.cc:407] Connection header received
0929 10:33:34.291693 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:34.291697 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:34.291766 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:34.291889 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:33:34.292127 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:34.292920 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:34.294010 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:34.294710 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:34.294821 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:34.294975 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:34.295113 (+ 138us) server_negotiation.cc:299] Negotiation successful
0929 10:33:34.295187 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260929 10:33:34.295964 22599 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 9df9b9ceadd74c56bc5c0242855d0777 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260929 10:33:34.296372 22628 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9df9b9ceadd74c56bc5c0242855d0777 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b919a4b6dcc4702b515509aa1dd2164. Latest consensus state: current_term: 1 leader_uuid: "2b919a4b6dcc4702b515509aa1dd2164" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2b919a4b6dcc4702b515509aa1dd2164" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 42245 } } peers { permanent_uuid: "ce1288d49e714ca69abeca37c76aebae" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 41741 } attrs { promote: false } } peers { permanent_uuid: "9df9b9ceadd74c56bc5c0242855d0777" member_type: NON_VOTER last_known_addr { host: "127.15.82.124" port: 41423 } attrs { promote: true } } }
I20260929 10:33:34.296525 22628 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9df9b9ceadd74c56bc5c0242855d0777 [sys.catalog]: This master's current role is: LEARNER
W20260929 10:33:34.310875 22912 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:33:34.311127 22912 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:33:34.311195 22912 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:33:34.315359 22912 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:33:34.315454 22912 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:33:34.315548 22912 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.67
I20260929 10:33:34.320798 22912 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:40517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/krb5kdc/oryx_127.15.82.67.keytab
--principal=oryx/127.15.82.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.67:43005
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.82.67
--webserver_port=34921
--tserver_master_addrs=127.15.82.126:42245,127.15.82.125:41741,127.15.82.124:41423
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22912
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.67
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:33:34.321792 22912 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:33:34.322881 22912 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:33:34.332624 22921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:34.332579 22919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:34.332620 22918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:34.333148 22912 server_base.cc:1034] running on GCE node
I20260929 10:33:34.333560 22912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:33:34.334226 22912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:33:34.335415 22912 hybrid_clock.cc:648] HybridClock initialized: now 1790678014335399 us; error 49 us; skew 500 ppm
Sep 29 10:33:34 dist-test-slave-vg89 krb5kdc[21759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678014, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.82.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:33:34.338223 22912 init.cc:377] Logged in from keytab as oryx/127.15.82.67@KRBTEST.COM (short username oryx)
I20260929 10:33:34.339215 22912 webserver.cc:460] Webserver started at http://127.15.82.67:34921/ using document root <none> and password file <none>
I20260929 10:33:34.339743 22912 fs_manager.cc:359] Metadata directory not provided
I20260929 10:33:34.339819 22912 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:33:34.343748 22912 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260929 10:33:34.345750 22928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:34.347015 22912 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 10:33:34.347131 22912 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-2/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-2/wal
uuid: "d2e85165e3af490ea649ff60a3d44c65"
format_stamp: "Formatted at 2026-09-29 10:33:33 on dist-test-slave-vg89"
server_key: "e808de97ab18ca7082b0764d9ddd001f"
server_key_iv: "b781639d5ba09f322a4583e3f9d31d6b"
server_key_version: "encryptionkey@0"
I20260929 10:33:34.347640 22912 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:33:34.365367 22912 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:33:34.368872 22912 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:33:34.369092 22912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:33:34.369769 22912 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:33:34.370718 22912 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:33:34.370790 22912 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:34.370860 22912 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:33:34.370910 22912 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:34.380566 22912 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.67:43005
I20260929 10:33:34.380592 23041 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.67:43005 every 8 connection(s)
I20260929 10:33:34.381867 22912 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
I20260929 10:33:34.386675 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 22912
Sep 29 10:33:34 dist-test-slave-vg89 krb5kdc[21759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678014, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.82.67@KRBTEST.COM for oryx/127.15.82.124@KRBTEST.COM
Sep 29 10:33:34 dist-test-slave-vg89 krb5kdc[21759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678014, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.82.67@KRBTEST.COM for oryx/127.15.82.125@KRBTEST.COM
Sep 29 10:33:34 dist-test-slave-vg89 krb5kdc[21759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678014, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.82.67@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM
I20260929 10:33:34.402527 22528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:34.385800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.67:41045 (local address 127.15.82.125:41741)
0929 10:33:34.386001 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:34.386007 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:34.386199 (+ 192us) server_negotiation.cc:407] Connection header received
0929 10:33:34.386820 (+ 621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:34.386823 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:34.386880 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:34.386936 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:34.389344 (+ 2408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:34.390167 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:34.392193 (+ 2026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:34.392540 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:34.392626 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:34.398269 (+ 5643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:34.398272 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:34.398275 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:34.398301 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:34.399911 (+ 1610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:34.400618 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:34.400624 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:34.400626 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:34.400715 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:34.401124 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:34.401127 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:34.401128 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:34.401348 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:34.401433 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:34.402171 (+ 738us) server_negotiation.cc:299] Negotiation successful
0929 10:33:34.402291 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260929 10:33:34.403680 22772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:34.385249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.67:57853 (local address 127.15.82.124:41423)
0929 10:33:34.385360 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:34.385364 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:34.385760 (+ 396us) server_negotiation.cc:407] Connection header received
0929 10:33:34.386919 (+ 1159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:34.386925 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:34.386977 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:34.387048 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:34.389344 (+ 2296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:34.390166 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:34.392216 (+ 2050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:34.392545 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:34.392621 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:34.396731 (+ 4110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:34.396735 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:34.396738 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:34.396764 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:34.401328 (+ 4564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:34.401841 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:34.401845 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:34.401846 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:34.401943 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:34.402245 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:34.402248 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:34.402249 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:34.402701 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:34.402787 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:34.403397 (+ 610us) server_negotiation.cc:299] Negotiation successful
0929 10:33:34.403513 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260929 10:33:34.404395 21843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:34.383829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.67:59719 (local address 127.15.82.126:42245)
0929 10:33:34.384003 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:34.384006 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:34.385369 (+ 1363us) server_negotiation.cc:407] Connection header received
0929 10:33:34.386903 (+ 1534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:34.386907 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:34.386972 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:34.387035 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:34.390598 (+ 3563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:34.391560 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:34.392699 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:34.393013 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:34.393087 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:34.397439 (+ 4352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:34.397442 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:34.397445 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:34.397472 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:34.400094 (+ 2622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:34.402032 (+ 1938us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:34.402037 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:34.402038 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:34.402117 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:34.402442 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:34.402447 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:34.402449 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:34.402685 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:34.402775 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:34.404145 (+ 1370us) server_negotiation.cc:299] Negotiation successful
0929 10:33:34.404248 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260929 10:33:34.404405 23053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:34.389143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38108 (local address 127.15.82.124:41423)
0929 10:33:34.392614 (+ 3471us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:34.392618 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:34.392629 (+ 11us) server_negotiation.cc:407] Connection header received
0929 10:33:34.392660 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:34.392663 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:34.392703 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:34.393350 (+ 647us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:34.394275 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:34.395057 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:34.396370 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:34.396753 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:34.396851 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:34.398885 (+ 2034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:34.398890 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:34.398893 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:34.398922 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:34.401360 (+ 2438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:34.401830 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:34.401834 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:34.401836 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:34.402942 (+ 1106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:34.403270 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:34.403273 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:34.403274 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:34.403461 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:34.403576 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:34.404116 (+ 540us) server_negotiation.cc:299] Negotiation successful
0929 10:33:34.404246 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":3374,"thread_start_us":90,"threads_started":1}
I20260929 10:33:34.405244 23046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:34.384211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:42245 (local address 127.15.82.67:59719)
0929 10:33:34.385192 (+ 981us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:34.385255 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:34.386572 (+ 1317us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:34.387318 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:34.387328 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:34.388171 (+ 843us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:34.390451 (+ 2280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:34.390458 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:34.391721 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:34.391724 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:34.392557 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:34.392569 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:34.393248 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:34.393254 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:34.393844 (+ 590us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:34.395250 (+ 1406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:34.395265 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:34.397206 (+ 1941us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:34.400860 (+ 3654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:34.400864 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:34.400867 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:34.401921 (+ 1054us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:34.402226 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:34.402232 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:34.402233 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:34.402314 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:34.402831 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:34.402833 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:34.403945 (+ 1112us) client_negotiation.cc:769] Sending connection context
0929 10:33:34.404529 (+ 584us) client_negotiation.cc:240] Negotiation successful
0929 10:33:34.404658 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":753,"mutex_wait_us":343,"spinlock_wait_cycles":4864,"thread_start_us":601,"threads_started":1}
I20260929 10:33:34.405248 23047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:34.385135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.124:41423 (local address 127.15.82.67:57853)
0929 10:33:34.385664 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:34.385679 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:34.386577 (+ 898us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:34.387317 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:34.387327 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:34.388112 (+ 785us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:34.389167 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:34.389189 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:34.390343 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:34.390346 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:34.392056 (+ 1710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:34.392069 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:34.392823 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:34.392828 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:34.393094 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:34.393841 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:34.393874 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:34.396544 (+ 2670us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:34.401452 (+ 4908us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:34.401455 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:34.401457 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:34.401724 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:34.402048 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:34.402051 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:34.402053 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:34.402140 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:34.402831 (+ 691us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:34.402834 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:34.402976 (+ 142us) client_negotiation.cc:769] Sending connection context
0929 10:33:34.403071 (+ 95us) client_negotiation.cc:240] Negotiation successful
0929 10:33:34.404027 (+ 956us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"mutex_wait_us":911,"spinlock_wait_cycles":17536,"thread_start_us":105,"threads_started":1}
I20260929 10:33:34.406097 23048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:34.385697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.125:41741 (local address 127.15.82.67:41045)
0929 10:33:34.386099 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:34.386115 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:34.386577 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:34.387317 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:34.387327 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:34.388110 (+ 783us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:34.389167 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:34.389189 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:34.390343 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:34.390346 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:34.392045 (+ 1699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:34.392055 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:34.392823 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:34.392826 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:34.393084 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:34.393863 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:34.393888 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:34.398116 (+ 4228us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:34.400026 (+ 1910us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:34.400033 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:34.400061 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:34.400513 (+ 452us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:34.400835 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:34.400839 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:34.400841 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:34.401010 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:34.401507 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:34.401515 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:34.401870 (+ 355us) client_negotiation.cc:769] Sending connection context
0929 10:33:34.403000 (+ 1130us) client_negotiation.cc:240] Negotiation successful
0929 10:33:34.403448 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"mutex_wait_us":490,"spinlock_wait_cycles":11136,"thread_start_us":88,"threads_started":1}
I20260929 10:33:34.406538 23043 heartbeater.cc:347] Connected to a master server at 127.15.82.126:42245
I20260929 10:33:34.406576 23042 heartbeater.cc:347] Connected to a master server at 127.15.82.124:41423
I20260929 10:33:34.406785 23042 heartbeater.cc:464] Registering TS with master...
I20260929 10:33:34.407207 23044 heartbeater.cc:347] Connected to a master server at 127.15.82.125:41741
I20260929 10:33:34.407285 23044 heartbeater.cc:464] Registering TS with master...
I20260929 10:33:34.407405 23042 heartbeater.cc:511] Master 127.15.82.124:41423 requested a full tablet report, sending...
I20260929 10:33:34.407477 23044 heartbeater.cc:511] Master 127.15.82.125:41741 requested a full tablet report, sending...
I20260929 10:33:34.406785 23043 heartbeater.cc:464] Registering TS with master...
I20260929 10:33:34.408110 23043 heartbeater.cc:511] Master 127.15.82.126:42245 requested a full tablet report, sending...
I20260929 10:33:34.408221 23054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:34.389192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37082 (local address 127.15.82.126:42245)
0929 10:33:34.389605 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:34.389613 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:34.390961 (+ 1348us) server_negotiation.cc:407] Connection header received
0929 10:33:34.391308 (+ 347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:34.391311 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:34.391369 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:34.391431 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:34.396397 (+ 4966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:34.397200 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:34.399944 (+ 2744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:34.400286 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:34.400372 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:34.402456 (+ 2084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:34.402459 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:34.402462 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:34.402488 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:34.404169 (+ 1681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:34.406839 (+ 2670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:34.406842 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:34.406844 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:34.406935 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:34.407249 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:34.407253 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:34.407255 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:34.407455 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:34.407573 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:34.407917 (+ 344us) server_negotiation.cc:299] Negotiation successful
0929 10:33:34.408052 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":74,"threads_started":1}
I20260929 10:33:34.408792 22586 ts_manager.cc:194] Registered new tserver with Master: d2e85165e3af490ea649ff60a3d44c65 (127.15.82.67:43005)
I20260929 10:33:34.408846 21799 ts_manager.cc:194] Re-registered known tserver with Master: d2e85165e3af490ea649ff60a3d44c65 (127.15.82.67:43005)
I20260929 10:33:34.409055 22406 ts_manager.cc:194] Registered new tserver with Master: d2e85165e3af490ea649ff60a3d44c65 (127.15.82.67:43005)
I20260929 10:33:34.409547 21799 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.82.67@KRBTEST.COM'} at 127.15.82.67:59719
I20260929 10:33:34.411690 23057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:34.390013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42300 (local address 127.15.82.125:41741)
0929 10:33:34.390268 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:34.390273 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:34.394281 (+ 4008us) server_negotiation.cc:407] Connection header received
0929 10:33:34.394325 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:34.394328 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:34.394378 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:34.394475 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:34.395421 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:34.396245 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:34.399606 (+ 3361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:34.401006 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:34.401116 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:34.406207 (+ 5091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:34.406212 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:34.406216 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:34.406243 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:34.407998 (+ 1755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:34.408487 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:34.408490 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:34.408492 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:34.409919 (+ 1427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:34.410246 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:34.410251 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:34.410255 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:34.410523 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:34.410657 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:34.411281 (+ 624us) server_negotiation.cc:299] Negotiation successful
0929 10:33:34.411411 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"spinlock_wait_cycles":3328,"thread_start_us":78,"threads_started":1}
I20260929 10:33:34.411648 21799 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37082:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 10:33:34.414831 21799 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:33:34.417313 22364 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b919a4b6dcc4702b515509aa1dd2164 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2b919a4b6dcc4702b515509aa1dd2164" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2b919a4b6dcc4702b515509aa1dd2164" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 42245 } } peers { permanent_uuid: "ce1288d49e714ca69abeca37c76aebae" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 41741 } attrs { promote: false } } peers { permanent_uuid: "9df9b9ceadd74c56bc5c0242855d0777" member_type: NON_VOTER last_known_addr { host: "127.15.82.124" port: 41423 } attrs { promote: true } } }
I20260929 10:33:34.417447 22364 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b919a4b6dcc4702b515509aa1dd2164 [sys.catalog]: This master's current role is: LEADER
I20260929 10:33:34.421222 22364 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 2b919a4b6dcc4702b515509aa1dd2164: attempting to promote NON_VOTER 9df9b9ceadd74c56bc5c0242855d0777 to VOTER
I20260929 10:33:34.421557 22364 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2b919a4b6dcc4702b515509aa1dd2164 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2b919a4b6dcc4702b515509aa1dd2164" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 42245 } } peers { permanent_uuid: "ce1288d49e714ca69abeca37c76aebae" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 41741 } attrs { promote: false } } peers { permanent_uuid: "9df9b9ceadd74c56bc5c0242855d0777" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 41423 } attrs { promote: false } }
I20260929 10:33:34.422675 22416 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ce1288d49e714ca69abeca37c76aebae [term 1 FOLLOWER]: Refusing update from remote peer 2b919a4b6dcc4702b515509aa1dd2164: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260929 10:33:34.423082 23062 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2b919a4b6dcc4702b515509aa1dd2164 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce1288d49e714ca69abeca37c76aebae" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 41741 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260929 10:33:34.423213 22598 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9df9b9ceadd74c56bc5c0242855d0777 [term 1 LEARNER]: Refusing update from remote peer 2b919a4b6dcc4702b515509aa1dd2164: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260929 10:33:34.423800 23062 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2b919a4b6dcc4702b515509aa1dd2164 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9df9b9ceadd74c56bc5c0242855d0777" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 41423 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260929 10:33:34.425089 23061 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2b919a4b6dcc4702b515509aa1dd2164 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 9df9b9ceadd74c56bc5c0242855d0777 (127.15.82.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2b919a4b6dcc4702b515509aa1dd2164" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 42245 } } peers { permanent_uuid: "ce1288d49e714ca69abeca37c76aebae" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 41741 } attrs { promote: false } } peers { permanent_uuid: "9df9b9ceadd74c56bc5c0242855d0777" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 41423 } attrs { promote: false } } }
I20260929 10:33:34.425813 22416 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ce1288d49e714ca69abeca37c76aebae [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 9df9b9ceadd74c56bc5c0242855d0777 (127.15.82.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2b919a4b6dcc4702b515509aa1dd2164" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 42245 } } peers { permanent_uuid: "ce1288d49e714ca69abeca37c76aebae" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 41741 } attrs { promote: false } } peers { permanent_uuid: "9df9b9ceadd74c56bc5c0242855d0777" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 41423 } attrs { promote: false } } }
I20260929 10:33:34.426653 23061 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b919a4b6dcc4702b515509aa1dd2164 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "2b919a4b6dcc4702b515509aa1dd2164" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2b919a4b6dcc4702b515509aa1dd2164" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 42245 } } peers { permanent_uuid: "ce1288d49e714ca69abeca37c76aebae" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 41741 } attrs { promote: false } } peers { permanent_uuid: "9df9b9ceadd74c56bc5c0242855d0777" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 41423 } attrs { promote: false } } }
I20260929 10:33:34.426751 23061 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b919a4b6dcc4702b515509aa1dd2164 [sys.catalog]: This master's current role is: LEADER
I20260929 10:33:34.426798 23062 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b919a4b6dcc4702b515509aa1dd2164 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2b919a4b6dcc4702b515509aa1dd2164" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2b919a4b6dcc4702b515509aa1dd2164" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 42245 } } peers { permanent_uuid: "ce1288d49e714ca69abeca37c76aebae" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 41741 } attrs { promote: false } } peers { permanent_uuid: "9df9b9ceadd74c56bc5c0242855d0777" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 41423 } attrs { promote: false } } }
I20260929 10:33:34.426909 23062 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b919a4b6dcc4702b515509aa1dd2164 [sys.catalog]: This master's current role is: LEADER
I20260929 10:33:34.427726 23070 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce1288d49e714ca69abeca37c76aebae [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2b919a4b6dcc4702b515509aa1dd2164" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2b919a4b6dcc4702b515509aa1dd2164" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 42245 } } peers { permanent_uuid: "ce1288d49e714ca69abeca37c76aebae" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 41741 } attrs { promote: false } } peers { permanent_uuid: "9df9b9ceadd74c56bc5c0242855d0777" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 41423 } attrs { promote: false } } }
I20260929 10:33:34.427857 23070 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce1288d49e714ca69abeca37c76aebae [sys.catalog]: This master's current role is: FOLLOWER
I20260929 10:33:34.428403 23062 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b919a4b6dcc4702b515509aa1dd2164 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2b919a4b6dcc4702b515509aa1dd2164" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2b919a4b6dcc4702b515509aa1dd2164" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 42245 } } peers { permanent_uuid: "ce1288d49e714ca69abeca37c76aebae" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 41741 } attrs { promote: false } } peers { permanent_uuid: "9df9b9ceadd74c56bc5c0242855d0777" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 41423 } attrs { promote: false } } }
I20260929 10:33:34.428503 23062 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b919a4b6dcc4702b515509aa1dd2164 [sys.catalog]: This master's current role is: LEADER
I20260929 10:33:34.428690 22598 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9df9b9ceadd74c56bc5c0242855d0777 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 9df9b9ceadd74c56bc5c0242855d0777 (127.15.82.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2b919a4b6dcc4702b515509aa1dd2164" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 42245 } } peers { permanent_uuid: "ce1288d49e714ca69abeca37c76aebae" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 41741 } attrs { promote: false } } peers { permanent_uuid: "9df9b9ceadd74c56bc5c0242855d0777" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 41423 } attrs { promote: false } } }
I20260929 10:33:34.430269 22628 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9df9b9ceadd74c56bc5c0242855d0777 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2b919a4b6dcc4702b515509aa1dd2164" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2b919a4b6dcc4702b515509aa1dd2164" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 42245 } } peers { permanent_uuid: "ce1288d49e714ca69abeca37c76aebae" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 41741 } attrs { promote: false } } peers { permanent_uuid: "9df9b9ceadd74c56bc5c0242855d0777" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 41423 } attrs { promote: false } } }
I20260929 10:33:34.430392 22628 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9df9b9ceadd74c56bc5c0242855d0777 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 10:33:34.437306 23071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:34.432012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.65:45833 (local address 127.0.0.1:33390)
0929 10:33:34.432323 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:34.432338 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:34.432426 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:34.433069 (+ 643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:34.433075 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:34.433093 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:33:34.433214 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:34.433221 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:34.434319 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:34.434322 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:34.435626 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:34.435637 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:34.436904 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:34.436907 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:34.436995 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:34.437023 (+ 28us) client_negotiation.cc:769] Sending connection context
0929 10:33:34.437067 (+ 44us) client_negotiation.cc:240] Negotiation successful
0929 10:33:34.437140 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":140,"threads_started":1}
I20260929 10:33:34.437935 23072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:34.432206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33390 (local address 127.15.82.65:45833)
0929 10:33:34.432510 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:34.432529 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:34.432557 (+ 28us) server_negotiation.cc:407] Connection header received
0929 10:33:34.432641 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:34.432647 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:34.432896 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:34.433108 (+ 212us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:33:34.433312 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:34.434219 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:34.435765 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:34.436771 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:34.437488 (+ 717us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:34.437661 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:34.437713 (+ 52us) server_negotiation.cc:299] Negotiation successful
0929 10:33:34.437799 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":80,"threads_started":1}
I20260929 10:33:34.438702 22913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:34.431720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.66:43097 (local address 127.0.0.1:41148)
0929 10:33:34.431863 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:34.431882 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:34.431961 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:34.434644 (+ 2683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:34.434647 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:34.434666 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:33:34.434757 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:34.434762 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:34.436091 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:34.436094 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:34.437185 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:34.437193 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:34.438304 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:34.438307 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:34.438387 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:34.438430 (+ 43us) client_negotiation.cc:769] Sending connection context
0929 10:33:34.438516 (+ 86us) client_negotiation.cc:240] Negotiation successful
0929 10:33:34.438593 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260929 10:33:34.438763 23075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:34.432988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41148 (local address 127.15.82.66:43097)
0929 10:33:34.433879 (+ 891us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:34.433892 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:34.433915 (+ 23us) server_negotiation.cc:407] Connection header received
0929 10:33:34.434216 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:34.434221 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:34.434498 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:34.434654 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:33:34.434880 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:34.435947 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:34.437302 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:34.438139 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:34.438300 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:34.438461 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:34.438543 (+ 82us) server_negotiation.cc:299] Negotiation successful
0929 10:33:34.438630 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":791,"thread_start_us":118,"threads_started":1}
I20260929 10:33:34.439303 23074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:34.432785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45642 (local address 127.15.82.67:43005)
0929 10:33:34.434668 (+ 1883us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:34.434680 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:34.434701 (+ 21us) server_negotiation.cc:407] Connection header received
0929 10:33:34.434773 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:34.434780 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:34.434972 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:34.435185 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:33:34.435507 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:34.436289 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:34.437398 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:34.438274 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:34.438367 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:34.439018 (+ 651us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:34.439064 (+ 46us) server_negotiation.cc:299] Negotiation successful
0929 10:33:34.439178 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1769,"thread_start_us":114,"threads_started":1}
I20260929 10:33:34.439473 23073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:34.432617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.67:43005 (local address 127.0.0.1:45642)
0929 10:33:34.432836 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:34.432849 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:34.432932 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:34.435310 (+ 2378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:34.435312 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:34.435326 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:33:34.435390 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:34.435393 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:34.436411 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:34.436414 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:34.437268 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:34.437281 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:34.438488 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:34.438492 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:34.438572 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:34.438593 (+ 21us) client_negotiation.cc:769] Sending connection context
0929 10:33:34.438651 (+ 58us) client_negotiation.cc:240] Negotiation successful
0929 10:33:34.438722 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":95,"threads_started":1}
I20260929 10:33:34.441251 22702 tablet_service.cc:1467] Processing CreateTablet for tablet b5ef4486efa64b4fb838bb310b2db5c6 (DEFAULT_TABLE table=test-table [id=891308b8fd0a48faa765aef10d771a2b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:33:34.441677 22839 tablet_service.cc:1467] Processing CreateTablet for tablet b5ef4486efa64b4fb838bb310b2db5c6 (DEFAULT_TABLE table=test-table [id=891308b8fd0a48faa765aef10d771a2b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:33:34.441905 22976 tablet_service.cc:1467] Processing CreateTablet for tablet b5ef4486efa64b4fb838bb310b2db5c6 (DEFAULT_TABLE table=test-table [id=891308b8fd0a48faa765aef10d771a2b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:33:34.442330 22702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5ef4486efa64b4fb838bb310b2db5c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:33:34.442736 22976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5ef4486efa64b4fb838bb310b2db5c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:33:34.442736 22839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5ef4486efa64b4fb838bb310b2db5c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:33:34.449261 23076 tablet_bootstrap.cc:492] T b5ef4486efa64b4fb838bb310b2db5c6 P 068171b1b6514121936f4634de5a8064: Bootstrap starting.
I20260929 10:33:34.450946 23077 tablet_bootstrap.cc:492] T b5ef4486efa64b4fb838bb310b2db5c6 P d2e85165e3af490ea649ff60a3d44c65: Bootstrap starting.
I20260929 10:33:34.452044 23076 tablet_bootstrap.cc:654] T b5ef4486efa64b4fb838bb310b2db5c6 P 068171b1b6514121936f4634de5a8064: Neither blocks nor log segments found. Creating new log.
I20260929 10:33:34.452898 23076 log.cc:824] T b5ef4486efa64b4fb838bb310b2db5c6 P 068171b1b6514121936f4634de5a8064: Log is configured to *not* fsync() on all Append() calls
I20260929 10:33:34.453313 23077 tablet_bootstrap.cc:654] T b5ef4486efa64b4fb838bb310b2db5c6 P d2e85165e3af490ea649ff60a3d44c65: Neither blocks nor log segments found. Creating new log.
I20260929 10:33:34.453605 23078 tablet_bootstrap.cc:492] T b5ef4486efa64b4fb838bb310b2db5c6 P 9886d2dce2354767a0ee1823c1021808: Bootstrap starting.
I20260929 10:33:34.454141 23077 log.cc:824] T b5ef4486efa64b4fb838bb310b2db5c6 P d2e85165e3af490ea649ff60a3d44c65: Log is configured to *not* fsync() on all Append() calls
I20260929 10:33:34.455118 23076 tablet_bootstrap.cc:492] T b5ef4486efa64b4fb838bb310b2db5c6 P 068171b1b6514121936f4634de5a8064: No bootstrap required, opened a new log
I20260929 10:33:34.455325 23076 ts_tablet_manager.cc:1397] T b5ef4486efa64b4fb838bb310b2db5c6 P 068171b1b6514121936f4634de5a8064: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.001s
I20260929 10:33:34.455729 23078 tablet_bootstrap.cc:654] T b5ef4486efa64b4fb838bb310b2db5c6 P 9886d2dce2354767a0ee1823c1021808: Neither blocks nor log segments found. Creating new log.
I20260929 10:33:34.456298 23077 tablet_bootstrap.cc:492] T b5ef4486efa64b4fb838bb310b2db5c6 P d2e85165e3af490ea649ff60a3d44c65: No bootstrap required, opened a new log
I20260929 10:33:34.456403 23078 log.cc:824] T b5ef4486efa64b4fb838bb310b2db5c6 P 9886d2dce2354767a0ee1823c1021808: Log is configured to *not* fsync() on all Append() calls
I20260929 10:33:34.456489 23077 ts_tablet_manager.cc:1397] T b5ef4486efa64b4fb838bb310b2db5c6 P d2e85165e3af490ea649ff60a3d44c65: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260929 10:33:34.457957 23078 tablet_bootstrap.cc:492] T b5ef4486efa64b4fb838bb310b2db5c6 P 9886d2dce2354767a0ee1823c1021808: No bootstrap required, opened a new log
I20260929 10:33:34.458151 23078 ts_tablet_manager.cc:1397] T b5ef4486efa64b4fb838bb310b2db5c6 P 9886d2dce2354767a0ee1823c1021808: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260929 10:33:34.460433 23076 raft_consensus.cc:348] T b5ef4486efa64b4fb838bb310b2db5c6 P 068171b1b6514121936f4634de5a8064 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9886d2dce2354767a0ee1823c1021808" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43097 } } peers { permanent_uuid: "068171b1b6514121936f4634de5a8064" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45833 } } peers { permanent_uuid: "d2e85165e3af490ea649ff60a3d44c65" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 43005 } }
I20260929 10:33:34.460762 23076 raft_consensus.cc:374] T b5ef4486efa64b4fb838bb310b2db5c6 P 068171b1b6514121936f4634de5a8064 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:33:34.460825 23076 raft_consensus.cc:729] T b5ef4486efa64b4fb838bb310b2db5c6 P 068171b1b6514121936f4634de5a8064 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 068171b1b6514121936f4634de5a8064, State: Initialized, Role: FOLLOWER
I20260929 10:33:34.461208 23077 raft_consensus.cc:348] T b5ef4486efa64b4fb838bb310b2db5c6 P d2e85165e3af490ea649ff60a3d44c65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9886d2dce2354767a0ee1823c1021808" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43097 } } peers { permanent_uuid: "068171b1b6514121936f4634de5a8064" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45833 } } peers { permanent_uuid: "d2e85165e3af490ea649ff60a3d44c65" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 43005 } }
I20260929 10:33:34.461311 23076 consensus_queue.cc:260] T b5ef4486efa64b4fb838bb310b2db5c6 P 068171b1b6514121936f4634de5a8064 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9886d2dce2354767a0ee1823c1021808" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43097 } } peers { permanent_uuid: "068171b1b6514121936f4634de5a8064" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45833 } } peers { permanent_uuid: "d2e85165e3af490ea649ff60a3d44c65" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 43005 } }
I20260929 10:33:34.461449 23077 raft_consensus.cc:374] T b5ef4486efa64b4fb838bb310b2db5c6 P d2e85165e3af490ea649ff60a3d44c65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:33:34.461521 23077 raft_consensus.cc:729] T b5ef4486efa64b4fb838bb310b2db5c6 P d2e85165e3af490ea649ff60a3d44c65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2e85165e3af490ea649ff60a3d44c65, State: Initialized, Role: FOLLOWER
I20260929 10:33:34.462450 23076 ts_tablet_manager.cc:1428] T b5ef4486efa64b4fb838bb310b2db5c6 P 068171b1b6514121936f4634de5a8064: Time spent starting tablet: real 0.007s user 0.003s sys 0.005s
I20260929 10:33:34.462497 22769 heartbeater.cc:503] Master 127.15.82.126:42245 was elected leader, sending a full tablet report...
I20260929 10:33:34.462980 23077 consensus_queue.cc:260] T b5ef4486efa64b4fb838bb310b2db5c6 P d2e85165e3af490ea649ff60a3d44c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9886d2dce2354767a0ee1823c1021808" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43097 } } peers { permanent_uuid: "068171b1b6514121936f4634de5a8064" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45833 } } peers { permanent_uuid: "d2e85165e3af490ea649ff60a3d44c65" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 43005 } }
I20260929 10:33:34.464012 23077 ts_tablet_manager.cc:1428] T b5ef4486efa64b4fb838bb310b2db5c6 P d2e85165e3af490ea649ff60a3d44c65: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260929 10:33:34.464946 23043 heartbeater.cc:503] Master 127.15.82.126:42245 was elected leader, sending a full tablet report...
I20260929 10:33:34.462925 23078 raft_consensus.cc:348] T b5ef4486efa64b4fb838bb310b2db5c6 P 9886d2dce2354767a0ee1823c1021808 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9886d2dce2354767a0ee1823c1021808" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43097 } } peers { permanent_uuid: "068171b1b6514121936f4634de5a8064" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45833 } } peers { permanent_uuid: "d2e85165e3af490ea649ff60a3d44c65" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 43005 } }
I20260929 10:33:34.466387 23078 raft_consensus.cc:374] T b5ef4486efa64b4fb838bb310b2db5c6 P 9886d2dce2354767a0ee1823c1021808 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:33:34.466471 23078 raft_consensus.cc:729] T b5ef4486efa64b4fb838bb310b2db5c6 P 9886d2dce2354767a0ee1823c1021808 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9886d2dce2354767a0ee1823c1021808, State: Initialized, Role: FOLLOWER
I20260929 10:33:34.466907 23078 consensus_queue.cc:260] T b5ef4486efa64b4fb838bb310b2db5c6 P 9886d2dce2354767a0ee1823c1021808 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9886d2dce2354767a0ee1823c1021808" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43097 } } peers { permanent_uuid: "068171b1b6514121936f4634de5a8064" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45833 } } peers { permanent_uuid: "d2e85165e3af490ea649ff60a3d44c65" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 43005 } }
I20260929 10:33:34.467600 22906 heartbeater.cc:503] Master 127.15.82.126:42245 was elected leader, sending a full tablet report...
I20260929 10:33:34.467911 23078 ts_tablet_manager.cc:1428] T b5ef4486efa64b4fb838bb310b2db5c6 P 9886d2dce2354767a0ee1823c1021808: Time spent starting tablet: real 0.010s user 0.006s sys 0.000s
W20260929 10:33:34.474586 22771 tablet.cc:2369] T b5ef4486efa64b4fb838bb310b2db5c6 P 068171b1b6514121936f4634de5a8064: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 10:33:34.548277 23084 raft_consensus.cc:482] T b5ef4486efa64b4fb838bb310b2db5c6 P 9886d2dce2354767a0ee1823c1021808 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:33:34.548487 23084 raft_consensus.cc:504] T b5ef4486efa64b4fb838bb310b2db5c6 P 9886d2dce2354767a0ee1823c1021808 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9886d2dce2354767a0ee1823c1021808" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43097 } } peers { permanent_uuid: "068171b1b6514121936f4634de5a8064" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45833 } } peers { permanent_uuid: "d2e85165e3af490ea649ff60a3d44c65" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 43005 } }
I20260929 10:33:34.549724 23084 leader_election.cc:288] T b5ef4486efa64b4fb838bb310b2db5c6 P 9886d2dce2354767a0ee1823c1021808 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 068171b1b6514121936f4634de5a8064 (127.15.82.65:45833), d2e85165e3af490ea649ff60a3d44c65 (127.15.82.67:43005)
I20260929 10:33:34.553966 23074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:34.550259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:52905 (local address 127.15.82.67:43005)
0929 10:33:34.550377 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:34.550381 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:34.550416 (+ 35us) server_negotiation.cc:407] Connection header received
0929 10:33:34.550474 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:34.550477 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:34.550527 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:34.550612 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:33:34.550947 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:34.551731 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:34.552982 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:34.553402 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:34.553529 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:34.553595 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:34.553777 (+ 182us) server_negotiation.cc:299] Negotiation successful
0929 10:33:34.553850 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260929 10:33:34.553978 22910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:34.549960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.67:43005 (local address 127.15.82.66:52905)
0929 10:33:34.550142 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:34.550160 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:34.550263 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:34.550664 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:34.550667 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:34.550685 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:33:34.550816 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:34.550823 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:34.551837 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:34.551843 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:34.552859 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:34.552868 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:34.553523 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:34.553527 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:34.553632 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:34.553657 (+ 25us) client_negotiation.cc:769] Sending connection context
0929 10:33:34.553748 (+ 91us) client_negotiation.cc:240] Negotiation successful
0929 10:33:34.553827 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80,"mutex_wait_us":1}
I20260929 10:33:34.554737 22996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5ef4486efa64b4fb838bb310b2db5c6" candidate_uuid: "9886d2dce2354767a0ee1823c1021808" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2e85165e3af490ea649ff60a3d44c65" is_pre_election: true
I20260929 10:33:34.554925 23072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:34.550205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:42443 (local address 127.15.82.65:45833)
0929 10:33:34.550362 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:34.550369 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:34.550745 (+ 376us) server_negotiation.cc:407] Connection header received
0929 10:33:34.550814 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:34.550816 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:34.550859 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:34.550952 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:33:34.551305 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:34.552071 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:34.553511 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:34.554015 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:34.554175 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:34.554240 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:34.554729 (+ 489us) server_negotiation.cc:299] Negotiation successful
0929 10:33:34.554812 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260929 10:33:34.554925 22911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:34.549969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.65:45833 (local address 127.15.82.66:42443)
0929 10:33:34.550529 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:34.550542 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:34.550650 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:34.550966 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:34.550970 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:34.550986 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:33:34.551155 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:34.551167 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:34.552185 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:34.552192 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:34.553352 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:34.553366 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:34.554483 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:34.554487 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:34.554592 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:34.554611 (+ 19us) client_negotiation.cc:769] Sending connection context
0929 10:33:34.554709 (+ 98us) client_negotiation.cc:240] Negotiation successful
0929 10:33:34.554784 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"mutex_wait_us":376}
I20260929 10:33:34.555115 22996 raft_consensus.cc:2454] T b5ef4486efa64b4fb838bb310b2db5c6 P d2e85165e3af490ea649ff60a3d44c65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9886d2dce2354767a0ee1823c1021808 in term 0.
I20260929 10:33:34.555635 22794 leader_election.cc:302] T b5ef4486efa64b4fb838bb310b2db5c6 P 9886d2dce2354767a0ee1823c1021808 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9886d2dce2354767a0ee1823c1021808, d2e85165e3af490ea649ff60a3d44c65; no voters:
I20260929 10:33:34.555631 22722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5ef4486efa64b4fb838bb310b2db5c6" candidate_uuid: "9886d2dce2354767a0ee1823c1021808" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "068171b1b6514121936f4634de5a8064" is_pre_election: true
I20260929 10:33:34.555902 22722 raft_consensus.cc:2454] T b5ef4486efa64b4fb838bb310b2db5c6 P 068171b1b6514121936f4634de5a8064 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9886d2dce2354767a0ee1823c1021808 in term 0.
I20260929 10:33:34.555932 23084 raft_consensus.cc:2781] T b5ef4486efa64b4fb838bb310b2db5c6 P 9886d2dce2354767a0ee1823c1021808 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:33:34.556104 23084 raft_consensus.cc:482] T b5ef4486efa64b4fb838bb310b2db5c6 P 9886d2dce2354767a0ee1823c1021808 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:33:34.556180 23084 raft_consensus.cc:3037] T b5ef4486efa64b4fb838bb310b2db5c6 P 9886d2dce2354767a0ee1823c1021808 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:33:34.557341 23084 raft_consensus.cc:504] T b5ef4486efa64b4fb838bb310b2db5c6 P 9886d2dce2354767a0ee1823c1021808 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9886d2dce2354767a0ee1823c1021808" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43097 } } peers { permanent_uuid: "068171b1b6514121936f4634de5a8064" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45833 } } peers { permanent_uuid: "d2e85165e3af490ea649ff60a3d44c65" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 43005 } }
I20260929 10:33:34.557715 23084 leader_election.cc:288] T b5ef4486efa64b4fb838bb310b2db5c6 P 9886d2dce2354767a0ee1823c1021808 [CANDIDATE]: Term 1 election: Requested vote from peers 068171b1b6514121936f4634de5a8064 (127.15.82.65:45833), d2e85165e3af490ea649ff60a3d44c65 (127.15.82.67:43005)
I20260929 10:33:34.558086 22722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5ef4486efa64b4fb838bb310b2db5c6" candidate_uuid: "9886d2dce2354767a0ee1823c1021808" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "068171b1b6514121936f4634de5a8064"
I20260929 10:33:34.558198 22996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5ef4486efa64b4fb838bb310b2db5c6" candidate_uuid: "9886d2dce2354767a0ee1823c1021808" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2e85165e3af490ea649ff60a3d44c65"
I20260929 10:33:34.558208 22722 raft_consensus.cc:3037] T b5ef4486efa64b4fb838bb310b2db5c6 P 068171b1b6514121936f4634de5a8064 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:33:34.558336 22996 raft_consensus.cc:3037] T b5ef4486efa64b4fb838bb310b2db5c6 P d2e85165e3af490ea649ff60a3d44c65 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:33:34.559319 22722 raft_consensus.cc:2454] T b5ef4486efa64b4fb838bb310b2db5c6 P 068171b1b6514121936f4634de5a8064 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9886d2dce2354767a0ee1823c1021808 in term 1.
I20260929 10:33:34.559479 22996 raft_consensus.cc:2454] T b5ef4486efa64b4fb838bb310b2db5c6 P d2e85165e3af490ea649ff60a3d44c65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9886d2dce2354767a0ee1823c1021808 in term 1.
I20260929 10:33:34.559698 22792 leader_election.cc:302] T b5ef4486efa64b4fb838bb310b2db5c6 P 9886d2dce2354767a0ee1823c1021808 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068171b1b6514121936f4634de5a8064, 9886d2dce2354767a0ee1823c1021808; no voters:
I20260929 10:33:34.559913 23084 raft_consensus.cc:2781] T b5ef4486efa64b4fb838bb310b2db5c6 P 9886d2dce2354767a0ee1823c1021808 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:33:34.560144 23084 raft_consensus.cc:686] T b5ef4486efa64b4fb838bb310b2db5c6 P 9886d2dce2354767a0ee1823c1021808 [term 1 LEADER]: Becoming Leader. State: Replica: 9886d2dce2354767a0ee1823c1021808, State: Running, Role: LEADER
I20260929 10:33:34.560518 23084 consensus_queue.cc:237] T b5ef4486efa64b4fb838bb310b2db5c6 P 9886d2dce2354767a0ee1823c1021808 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9886d2dce2354767a0ee1823c1021808" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43097 } } peers { permanent_uuid: "068171b1b6514121936f4634de5a8064" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45833 } } peers { permanent_uuid: "d2e85165e3af490ea649ff60a3d44c65" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 43005 } }
I20260929 10:33:34.564631 21804 catalog_manager.cc:5495] T b5ef4486efa64b4fb838bb310b2db5c6 P 9886d2dce2354767a0ee1823c1021808 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9886d2dce2354767a0ee1823c1021808 (127.15.82.66). New cstate: current_term: 1 leader_uuid: "9886d2dce2354767a0ee1823c1021808" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9886d2dce2354767a0ee1823c1021808" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "068171b1b6514121936f4634de5a8064" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45833 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2e85165e3af490ea649ff60a3d44c65" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 43005 } health_report { overall_health: UNKNOWN } } }
I20260929 10:33:34.588997 23075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:34.585160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41164 (local address 127.15.82.66:43097)
0929 10:33:34.585275 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:34.585278 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:34.585292 (+ 14us) server_negotiation.cc:407] Connection header received
0929 10:33:34.585421 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:34.585423 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:34.585478 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:34.585577 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:33:34.585778 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:34.586321 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:34.587403 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:34.587668 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:34.587750 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:34.588074 (+ 324us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:33:34.588661 (+ 587us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:33:34.588768 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:34.588825 (+ 57us) server_negotiation.cc:299] Negotiation successful
0929 10:33:34.588881 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260929 10:33:34.595224 22722 raft_consensus.cc:1261] T b5ef4486efa64b4fb838bb310b2db5c6 P 068171b1b6514121936f4634de5a8064 [term 1 FOLLOWER]: Refusing update from remote peer 9886d2dce2354767a0ee1823c1021808: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:33:34.595224 22996 raft_consensus.cc:1261] T b5ef4486efa64b4fb838bb310b2db5c6 P d2e85165e3af490ea649ff60a3d44c65 [term 1 FOLLOWER]: Refusing update from remote peer 9886d2dce2354767a0ee1823c1021808: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:33:34.595956 23084 consensus_queue.cc:1035] T b5ef4486efa64b4fb838bb310b2db5c6 P 9886d2dce2354767a0ee1823c1021808 [LEADER]: Connected to new peer: Peer: permanent_uuid: "068171b1b6514121936f4634de5a8064" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45833 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:33:34.596177 23085 consensus_queue.cc:1035] T b5ef4486efa64b4fb838bb310b2db5c6 P 9886d2dce2354767a0ee1823c1021808 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e85165e3af490ea649ff60a3d44c65" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 43005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:33:34.607651 23090 mvcc.cc:205] Tried to move back new op lower bound from 7334617147774332928 to 7334617147643887616. Current Snapshot: MvccSnapshot[applied={T|T < 7334617147774332928}]
I20260929 10:33:34.613667 21804 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37082:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:33:34.613813 21804 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37082:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:33:34.618110 21804 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2b919a4b6dcc4702b515509aa1dd2164: Sending DeleteTablet for 3 replicas of tablet b5ef4486efa64b4fb838bb310b2db5c6
I20260929 10:33:34.619040 22976 tablet_service.cc:1514] Processing DeleteTablet for tablet b5ef4486efa64b4fb838bb310b2db5c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:33:34 UTC) from {username='oryx', principal='oryx/127.15.82.126@KRBTEST.COM'} at 127.0.0.1:45642
I20260929 10:33:34.619243 22839 tablet_service.cc:1514] Processing DeleteTablet for tablet b5ef4486efa64b4fb838bb310b2db5c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:33:34 UTC) from {username='oryx', principal='oryx/127.15.82.126@KRBTEST.COM'} at 127.0.0.1:41148
I20260929 10:33:34.619355 22702 tablet_service.cc:1514] Processing DeleteTablet for tablet b5ef4486efa64b4fb838bb310b2db5c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:33:34 UTC) from {username='oryx', principal='oryx/127.15.82.126@KRBTEST.COM'} at 127.0.0.1:33390
I20260929 10:33:34.619807 23098 tablet_replica.cc:330] T b5ef4486efa64b4fb838bb310b2db5c6 P 9886d2dce2354767a0ee1823c1021808: stopping tablet replica
I20260929 10:33:34.620157 23098 raft_consensus.cc:2229] T b5ef4486efa64b4fb838bb310b2db5c6 P 9886d2dce2354767a0ee1823c1021808 [term 1 LEADER]: Raft consensus shutting down.
I20260929 10:33:34.620574 23098 raft_consensus.cc:2258] T b5ef4486efa64b4fb838bb310b2db5c6 P 9886d2dce2354767a0ee1823c1021808 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:33:34.621456 23099 tablet_replica.cc:330] T b5ef4486efa64b4fb838bb310b2db5c6 P 068171b1b6514121936f4634de5a8064: stopping tablet replica
I20260929 10:33:34.621784 23099 raft_consensus.cc:2229] T b5ef4486efa64b4fb838bb310b2db5c6 P 068171b1b6514121936f4634de5a8064 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:33:34.622123 23099 raft_consensus.cc:2258] T b5ef4486efa64b4fb838bb310b2db5c6 P 068171b1b6514121936f4634de5a8064 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:33:34.622665 23098 ts_tablet_manager.cc:1905] T b5ef4486efa64b4fb838bb310b2db5c6 P 9886d2dce2354767a0ee1823c1021808: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:33:34.623467 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 21772
I20260929 10:33:34.623657 23099 ts_tablet_manager.cc:1905] T b5ef4486efa64b4fb838bb310b2db5c6 P 068171b1b6514121936f4634de5a8064: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:33:34.626649 23097 tablet_replica.cc:330] T b5ef4486efa64b4fb838bb310b2db5c6 P d2e85165e3af490ea649ff60a3d44c65: stopping tablet replica
I20260929 10:33:34.627004 23097 raft_consensus.cc:2229] T b5ef4486efa64b4fb838bb310b2db5c6 P d2e85165e3af490ea649ff60a3d44c65 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:33:34.627038 23098 ts_tablet_manager.cc:1918] T b5ef4486efa64b4fb838bb310b2db5c6 P 9886d2dce2354767a0ee1823c1021808: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:33:34.627144 23098 log.cc:1196] T b5ef4486efa64b4fb838bb310b2db5c6 P 9886d2dce2354767a0ee1823c1021808: Deleting WAL directory at /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-1/wal/wals/b5ef4486efa64b4fb838bb310b2db5c6
I20260929 10:33:34.627357 23097 raft_consensus.cc:2258] T b5ef4486efa64b4fb838bb310b2db5c6 P d2e85165e3af490ea649ff60a3d44c65 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:33:34.627477 23098 ts_tablet_manager.cc:1939] T b5ef4486efa64b4fb838bb310b2db5c6 P 9886d2dce2354767a0ee1823c1021808: Deleting consensus metadata
I20260929 10:33:34.628988 23097 ts_tablet_manager.cc:1905] T b5ef4486efa64b4fb838bb310b2db5c6 P d2e85165e3af490ea649ff60a3d44c65: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:33:34.631661 23097 ts_tablet_manager.cc:1918] T b5ef4486efa64b4fb838bb310b2db5c6 P d2e85165e3af490ea649ff60a3d44c65: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:33:34.631764 23097 log.cc:1196] T b5ef4486efa64b4fb838bb310b2db5c6 P d2e85165e3af490ea649ff60a3d44c65: Deleting WAL directory at /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-2/wal/wals/b5ef4486efa64b4fb838bb310b2db5c6
I20260929 10:33:34.632097 23097 ts_tablet_manager.cc:1939] T b5ef4486efa64b4fb838bb310b2db5c6 P d2e85165e3af490ea649ff60a3d44c65: Deleting consensus metadata
W20260929 10:33:34.633270 22930 connection.cc:441] server connection from 127.0.0.1:45642 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45642 (request call id 1) could send its response
I20260929 10:33:34.633649 23099 ts_tablet_manager.cc:1918] T b5ef4486efa64b4fb838bb310b2db5c6 P 068171b1b6514121936f4634de5a8064: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:33:34.633756 23099 log.cc:1196] T b5ef4486efa64b4fb838bb310b2db5c6 P 068171b1b6514121936f4634de5a8064: Deleting WAL directory at /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-0/wal/wals/b5ef4486efa64b4fb838bb310b2db5c6
I20260929 10:33:34.634068 23099 ts_tablet_manager.cc:1939] T b5ef4486efa64b4fb838bb310b2db5c6 P 068171b1b6514121936f4634de5a8064: Deleting consensus metadata
W20260929 10:33:34.634009 22795 connection.cc:535] server connection from 127.0.0.1:41148 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260929 10:33:34.635735 22655 connection.cc:441] server connection from 127.0.0.1:33390 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33390 (request call id 1) could send its response
I20260929 10:33:34.639786 23057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:34.621704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42304 (local address 127.15.82.125:41741)
0929 10:33:34.622248 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:34.622253 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:34.627988 (+ 5735us) server_negotiation.cc:407] Connection header received
0929 10:33:34.628233 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:34.628239 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:34.628307 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:34.628363 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:34.630753 (+ 2390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:34.631709 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:34.633203 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:34.633557 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:34.633660 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:34.635753 (+ 2093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:34.635759 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:34.635766 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:34.635802 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:34.637709 (+ 1907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:34.638238 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:34.638242 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:34.638244 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:34.638335 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:34.638684 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:34.638687 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:34.638689 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:34.638967 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:34.639113 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:34.639446 (+ 333us) server_negotiation.cc:299] Negotiation successful
0929 10:33:34.639573 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":456}
I20260929 10:33:34.651760 22528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:34.639203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42310 (local address 127.15.82.125:41741)
0929 10:33:34.639322 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:34.639326 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:34.639878 (+ 552us) server_negotiation.cc:407] Connection header received
0929 10:33:34.640039 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:34.640042 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:34.640095 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:34.640144 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:34.641693 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:34.642755 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:34.644442 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:34.644828 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:34.644913 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:34.647222 (+ 2309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:34.647226 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:34.647230 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:34.647259 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:34.649386 (+ 2127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:34.649985 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:34.649989 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:34.649993 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:34.650089 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:34.650530 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:34.650535 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:34.650537 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:34.650842 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:34.650957 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:34.651360 (+ 403us) server_negotiation.cc:299] Negotiation successful
0929 10:33:34.651506 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260929 10:33:34.652475 23053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:34.622391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38112 (local address 127.15.82.124:41423)
0929 10:33:34.622535 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:34.622540 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:34.639917 (+ 17377us) server_negotiation.cc:407] Connection header received
0929 10:33:34.639972 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:34.639975 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:34.640027 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:34.640117 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:34.641666 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:34.642752 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:34.644442 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:34.644828 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:34.644923 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:34.647222 (+ 2299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:34.647226 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:34.647230 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:34.647259 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:34.649548 (+ 2289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:34.650084 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:34.650088 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:34.650090 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:34.650181 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:34.650712 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:34.650715 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:34.650717 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:34.650938 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:34.651071 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:34.652167 (+ 1096us) server_negotiation.cc:299] Negotiation successful
0929 10:33:34.652318 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260929 10:33:34.652926 22772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:34.639273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38116 (local address 127.15.82.124:41423)
0929 10:33:34.639402 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:34.639407 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:34.642669 (+ 3262us) server_negotiation.cc:407] Connection header received
0929 10:33:34.642836 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:34.642839 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:34.642886 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:34.642936 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:34.643878 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:34.644661 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:34.646624 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:34.646950 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:34.647026 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:34.649032 (+ 2006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:34.649036 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:34.649040 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:34.649067 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:34.650856 (+ 1789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:34.651429 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:34.651432 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:34.651434 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:34.651515 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:34.651852 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:34.651855 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:34.651857 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:34.652064 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:34.652206 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:34.652648 (+ 442us) server_negotiation.cc:299] Negotiation successful
0929 10:33:34.652764 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260929 10:33:35.608646 23108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:35.608030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:42245 (local address 127.15.82.66:39829)
0929 10:33:35.608394 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:35.608494 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:42245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":77,"threads_started":1}
W20260929 10:33:35.609226 22906 heartbeater.cc:644] Failed to heartbeat to 127.15.82.126:42245 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.15.82.126:42245: connect: Connection refused (error 111)
I20260929 10:33:35.609611 23109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:35.609045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:42245 (local address 127.15.82.67:50199)
0929 10:33:35.609392 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:35.609508 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:42245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":76,"threads_started":1}
W20260929 10:33:35.610076 23043 heartbeater.cc:644] Failed to heartbeat to 127.15.82.126:42245 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.15.82.126:42245: connect: Connection refused (error 111)
I20260929 10:33:35.612594 23110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:35.612139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:42245 (local address 127.15.82.65:57725)
0929 10:33:35.612405 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:35.612494 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:42245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":58,"threads_started":1}
W20260929 10:33:35.612846 22769 heartbeater.cc:644] Failed to heartbeat to 127.15.82.126:42245 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.15.82.126:42245: connect: Connection refused (error 111)
I20260929 10:33:36.118831 23111 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9df9b9ceadd74c56bc5c0242855d0777 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2b919a4b6dcc4702b515509aa1dd2164)
I20260929 10:33:36.119100 23111 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9df9b9ceadd74c56bc5c0242855d0777 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2b919a4b6dcc4702b515509aa1dd2164" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 42245 } } peers { permanent_uuid: "ce1288d49e714ca69abeca37c76aebae" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 41741 } attrs { promote: false } } peers { permanent_uuid: "9df9b9ceadd74c56bc5c0242855d0777" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 41423 } attrs { promote: false } }
I20260929 10:33:36.120283 23111 leader_election.cc:288] T 00000000000000000000000000000000 P 9df9b9ceadd74c56bc5c0242855d0777 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2b919a4b6dcc4702b515509aa1dd2164 (127.15.82.126:42245), ce1288d49e714ca69abeca37c76aebae (127.15.82.125:41741)
I20260929 10:33:36.120836 23112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:36.120403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:42245 (local address 127.0.0.1:37566)
0929 10:33:36.120636 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:36.120708 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:42245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":74,"threads_started":1}
I20260929 10:33:36.120885 23113 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce1288d49e714ca69abeca37c76aebae [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2b919a4b6dcc4702b515509aa1dd2164)
W20260929 10:33:36.121064 22573 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.82.126:42245: connect: Connection refused (error 111)
I20260929 10:33:36.121037 23113 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce1288d49e714ca69abeca37c76aebae [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2b919a4b6dcc4702b515509aa1dd2164" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 42245 } } peers { permanent_uuid: "ce1288d49e714ca69abeca37c76aebae" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 41741 } attrs { promote: false } } peers { permanent_uuid: "9df9b9ceadd74c56bc5c0242855d0777" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 41423 } attrs { promote: false } }
I20260929 10:33:36.121980 23113 leader_election.cc:288] T 00000000000000000000000000000000 P ce1288d49e714ca69abeca37c76aebae [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2b919a4b6dcc4702b515509aa1dd2164 (127.15.82.126:42245), 9df9b9ceadd74c56bc5c0242855d0777 (127.15.82.124:41423)
I20260929 10:33:36.122349 23114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:36.121730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:42245 (local address 127.0.0.1:37584)
0929 10:33:36.122192 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:36.122264 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:42245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379}
W20260929 10:33:36.122480 22573 leader_election.cc:334] T 00000000000000000000000000000000 P 9df9b9ceadd74c56bc5c0242855d0777 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2b919a4b6dcc4702b515509aa1dd2164 (127.15.82.126:42245): Network error: Client connection negotiation failed: client connection to 127.15.82.126:42245: connect: Connection refused (error 111)
I20260929 10:33:36.122685 23117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:36.121927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:42245 (local address 127.0.0.1:37576)
0929 10:33:36.122375 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:36.122550 (+ 175us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:42245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":90,"threads_started":1}
W20260929 10:33:36.122841 22390 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.82.126:42245: connect: Connection refused (error 111)
I20260929 10:33:36.123754 23117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:36.123506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:42245 (local address 127.0.0.1:37590)
0929 10:33:36.123626 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:36.123684 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:42245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260929 10:33:36.123934 22390 leader_election.cc:334] T 00000000000000000000000000000000 P ce1288d49e714ca69abeca37c76aebae [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2b919a4b6dcc4702b515509aa1dd2164 (127.15.82.126:42245): Network error: Client connection negotiation failed: client connection to 127.15.82.126:42245: connect: Connection refused (error 111)
I20260929 10:33:36.125454 23112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:36.120807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.125:41741 (local address 127.0.0.1:42316)
0929 10:33:36.121045 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:36.121062 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:36.121274 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:36.121600 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:36.121604 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:36.121639 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:33:36.121802 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:36.121812 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:36.123024 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:36.123030 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:36.124280 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:36.124289 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:36.125165 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:36.125168 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:36.125233 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:36.125253 (+ 20us) client_negotiation.cc:769] Sending connection context
0929 10:33:36.125310 (+ 57us) client_negotiation.cc:240] Negotiation successful
0929 10:33:36.125362 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":28,"thread_start_us":53,"threads_started":1}
I20260929 10:33:36.125633 23115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:36.120911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42316 (local address 127.15.82.125:41741)
0929 10:33:36.121252 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:36.121256 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:36.121272 (+ 16us) server_negotiation.cc:407] Connection header received
0929 10:33:36.121419 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:36.121423 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:36.121472 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:36.121917 (+ 445us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:33:36.122067 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:36.122848 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:36.124401 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:36.125047 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:36.125170 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:36.125231 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:36.125383 (+ 152us) server_negotiation.cc:299] Negotiation successful
0929 10:33:36.125459 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":59,"threads_started":1}
I20260929 10:33:36.126089 22416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9df9b9ceadd74c56bc5c0242855d0777" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ce1288d49e714ca69abeca37c76aebae" is_pre_election: true
I20260929 10:33:36.126330 22416 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ce1288d49e714ca69abeca37c76aebae [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9df9b9ceadd74c56bc5c0242855d0777 in term 1.
I20260929 10:33:36.126778 23118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:36.121772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.124:41423 (local address 127.0.0.1:38126)
0929 10:33:36.122192 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:36.122208 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:36.122297 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:36.122702 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:36.122708 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:36.122731 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:33:36.122834 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:36.122844 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:36.124069 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:36.124073 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:36.125428 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:36.125437 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:36.126320 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:36.126324 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:36.126435 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:36.126462 (+ 27us) client_negotiation.cc:769] Sending connection context
0929 10:33:36.126535 (+ 73us) client_negotiation.cc:240] Negotiation successful
0929 10:33:36.126649 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"mutex_wait_us":1,"thread_start_us":65,"threads_started":1}
I20260929 10:33:36.126775 23119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:36.121920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38126 (local address 127.15.82.124:41423)
0929 10:33:36.122248 (+ 328us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:36.122256 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:36.122269 (+ 13us) server_negotiation.cc:407] Connection header received
0929 10:33:36.122447 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:36.122450 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:36.122496 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:36.122584 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:33:36.123050 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:36.123938 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:36.125649 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:36.126201 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:36.126321 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:36.126373 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:36.126562 (+ 189us) server_negotiation.cc:299] Negotiation successful
0929 10:33:36.126650 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":48,"server-negotiator.queue_time_us":268,"thread_start_us":117,"threads_started":1}
I20260929 10:33:36.126758 22573 leader_election.cc:302] T 00000000000000000000000000000000 P 9df9b9ceadd74c56bc5c0242855d0777 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9df9b9ceadd74c56bc5c0242855d0777, ce1288d49e714ca69abeca37c76aebae; no voters: 2b919a4b6dcc4702b515509aa1dd2164
I20260929 10:33:36.126997 23111 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9df9b9ceadd74c56bc5c0242855d0777 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260929 10:33:36.127051 23111 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9df9b9ceadd74c56bc5c0242855d0777 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2b919a4b6dcc4702b515509aa1dd2164)
I20260929 10:33:36.127094 23111 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9df9b9ceadd74c56bc5c0242855d0777 [term 1 FOLLOWER]: Advancing to term 2
I20260929 10:33:36.127254 22598 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ce1288d49e714ca69abeca37c76aebae" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "9df9b9ceadd74c56bc5c0242855d0777" is_pre_election: true
I20260929 10:33:36.128183 23111 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9df9b9ceadd74c56bc5c0242855d0777 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2b919a4b6dcc4702b515509aa1dd2164" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 42245 } } peers { permanent_uuid: "ce1288d49e714ca69abeca37c76aebae" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 41741 } attrs { promote: false } } peers { permanent_uuid: "9df9b9ceadd74c56bc5c0242855d0777" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 41423 } attrs { promote: false } }
I20260929 10:33:36.128439 22598 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 9df9b9ceadd74c56bc5c0242855d0777 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce1288d49e714ca69abeca37c76aebae in current term 2: Already voted for candidate 9df9b9ceadd74c56bc5c0242855d0777 in this term.
I20260929 10:33:36.128611 23111 leader_election.cc:288] T 00000000000000000000000000000000 P 9df9b9ceadd74c56bc5c0242855d0777 [CANDIDATE]: Term 2 election: Requested vote from peers 2b919a4b6dcc4702b515509aa1dd2164 (127.15.82.126:42245), ce1288d49e714ca69abeca37c76aebae (127.15.82.125:41741)
I20260929 10:33:36.128969 23112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:36.128723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:42245 (local address 127.0.0.1:37600)
0929 10:33:36.128838 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:36.128899 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:42245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260929 10:33:36.129110 22416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9df9b9ceadd74c56bc5c0242855d0777" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ce1288d49e714ca69abeca37c76aebae"
I20260929 10:33:36.129236 22416 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce1288d49e714ca69abeca37c76aebae [term 1 FOLLOWER]: Advancing to term 2
I20260929 10:33:36.129248 22392 leader_election.cc:302] T 00000000000000000000000000000000 P ce1288d49e714ca69abeca37c76aebae [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce1288d49e714ca69abeca37c76aebae; no voters: 2b919a4b6dcc4702b515509aa1dd2164, 9df9b9ceadd74c56bc5c0242855d0777
I20260929 10:33:36.129915 23112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:36.129679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:42245 (local address 127.0.0.1:37608)
0929 10:33:36.129808 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:36.129858 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:42245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260929 10:33:36.130061 22573 leader_election.cc:334] T 00000000000000000000000000000000 P 9df9b9ceadd74c56bc5c0242855d0777 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 2b919a4b6dcc4702b515509aa1dd2164 (127.15.82.126:42245): Network error: Client connection negotiation failed: client connection to 127.15.82.126:42245: connect: Connection refused (error 111)
I20260929 10:33:36.130319 22416 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ce1288d49e714ca69abeca37c76aebae [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9df9b9ceadd74c56bc5c0242855d0777 in term 2.
I20260929 10:33:36.130470 23113 raft_consensus.cc:2726] T 00000000000000000000000000000000 P ce1288d49e714ca69abeca37c76aebae [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 10:33:36.130656 22573 leader_election.cc:302] T 00000000000000000000000000000000 P 9df9b9ceadd74c56bc5c0242855d0777 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9df9b9ceadd74c56bc5c0242855d0777, ce1288d49e714ca69abeca37c76aebae; no voters: 2b919a4b6dcc4702b515509aa1dd2164
I20260929 10:33:36.130831 23111 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9df9b9ceadd74c56bc5c0242855d0777 [term 2 FOLLOWER]: Leader election won for term 2
I20260929 10:33:36.131021 23111 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9df9b9ceadd74c56bc5c0242855d0777 [term 2 LEADER]: Becoming Leader. State: Replica: 9df9b9ceadd74c56bc5c0242855d0777, State: Running, Role: LEADER
I20260929 10:33:36.131326 23111 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9df9b9ceadd74c56bc5c0242855d0777 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2b919a4b6dcc4702b515509aa1dd2164" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 42245 } } peers { permanent_uuid: "ce1288d49e714ca69abeca37c76aebae" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 41741 } attrs { promote: false } } peers { permanent_uuid: "9df9b9ceadd74c56bc5c0242855d0777" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 41423 } attrs { promote: false } }
I20260929 10:33:36.132412 23121 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9df9b9ceadd74c56bc5c0242855d0777 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9df9b9ceadd74c56bc5c0242855d0777. Latest consensus state: current_term: 2 leader_uuid: "9df9b9ceadd74c56bc5c0242855d0777" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2b919a4b6dcc4702b515509aa1dd2164" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 42245 } } peers { permanent_uuid: "ce1288d49e714ca69abeca37c76aebae" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 41741 } attrs { promote: false } } peers { permanent_uuid: "9df9b9ceadd74c56bc5c0242855d0777" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 41423 } attrs { promote: false } } }
I20260929 10:33:36.132503 23121 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9df9b9ceadd74c56bc5c0242855d0777 [sys.catalog]: This master's current role is: LEADER
I20260929 10:33:36.132766 23123 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:33:36.135063 23123 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:33:36.135501 23123 catalog_manager.cc:1259] Loaded cluster ID: 1f355f33ab054e5c8cc0fe255d71c834
I20260929 10:33:36.135576 23123 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:33:36.136161 23123 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:33:36.136588 23123 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9df9b9ceadd74c56bc5c0242855d0777: Loaded TSK: 0
I20260929 10:33:36.137293 23123 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 10:33:36.186141 22588 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38116:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 10:33:36.189522 22588 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:33:36.192646 23112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:36.192389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:42245 (local address 127.0.0.1:37612)
0929 10:33:36.192524 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:36.192578 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:42245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260929 10:33:36.192698 22416 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ce1288d49e714ca69abeca37c76aebae [term 2 FOLLOWER]: Refusing update from remote peer 9df9b9ceadd74c56bc5c0242855d0777: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260929 10:33:36.193269 23111 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9df9b9ceadd74c56bc5c0242855d0777 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce1288d49e714ca69abeca37c76aebae" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 41741 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260929 10:33:36.193605 23112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:36.193359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:42245 (local address 127.0.0.1:37616)
0929 10:33:36.193485 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:36.193544 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:42245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260929 10:33:36.193792 22573 consensus_peers.cc:487] T 00000000000000000000000000000000 P 9df9b9ceadd74c56bc5c0242855d0777 -> Peer 2b919a4b6dcc4702b515509aa1dd2164 (127.15.82.126:42245): Couldn't send request to peer 2b919a4b6dcc4702b515509aa1dd2164. Status: Network error: Client connection negotiation failed: client connection to 127.15.82.126:42245: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260929 10:33:36.194940 23113 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce1288d49e714ca69abeca37c76aebae [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9df9b9ceadd74c56bc5c0242855d0777. Latest consensus state: current_term: 2 leader_uuid: "9df9b9ceadd74c56bc5c0242855d0777" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2b919a4b6dcc4702b515509aa1dd2164" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 42245 } } peers { permanent_uuid: "ce1288d49e714ca69abeca37c76aebae" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 41741 } attrs { promote: false } } peers { permanent_uuid: "9df9b9ceadd74c56bc5c0242855d0777" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 41423 } attrs { promote: false } } }
I20260929 10:33:36.195067 23113 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce1288d49e714ca69abeca37c76aebae [sys.catalog]: This master's current role is: FOLLOWER
I20260929 10:33:36.196301 23121 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9df9b9ceadd74c56bc5c0242855d0777 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "9df9b9ceadd74c56bc5c0242855d0777" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2b919a4b6dcc4702b515509aa1dd2164" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 42245 } } peers { permanent_uuid: "ce1288d49e714ca69abeca37c76aebae" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 41741 } attrs { promote: false } } peers { permanent_uuid: "9df9b9ceadd74c56bc5c0242855d0777" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 41423 } attrs { promote: false } } }
I20260929 10:33:36.196425 23121 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9df9b9ceadd74c56bc5c0242855d0777 [sys.catalog]: This master's current role is: LEADER
I20260929 10:33:36.196586 23113 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce1288d49e714ca69abeca37c76aebae [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "9df9b9ceadd74c56bc5c0242855d0777" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2b919a4b6dcc4702b515509aa1dd2164" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 42245 } } peers { permanent_uuid: "ce1288d49e714ca69abeca37c76aebae" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 41741 } attrs { promote: false } } peers { permanent_uuid: "9df9b9ceadd74c56bc5c0242855d0777" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 41423 } attrs { promote: false } } }
I20260929 10:33:36.196712 23113 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce1288d49e714ca69abeca37c76aebae [sys.catalog]: This master's current role is: FOLLOWER
I20260929 10:33:36.206827 23114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:36.202619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.67:43005 (local address 127.0.0.1:45644)
0929 10:33:36.202772 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:36.202789 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:36.202884 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:36.203277 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:36.203280 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:36.203300 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:33:36.203421 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:36.203427 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:36.204560 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:36.204563 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:36.205401 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:36.205409 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:36.206360 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:36.206363 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:36.206508 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:36.206535 (+ 27us) client_negotiation.cc:769] Sending connection context
0929 10:33:36.206607 (+ 72us) client_negotiation.cc:240] Negotiation successful
0929 10:33:36.206671 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83,"mutex_wait_us":40}
I20260929 10:33:36.207089 23112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:36.202379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.65:45833 (local address 127.0.0.1:33394)
0929 10:33:36.202540 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:36.202556 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:36.202642 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:36.203415 (+ 773us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:36.203418 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:36.203435 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:33:36.203536 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:36.203544 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:36.204629 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:36.204632 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:36.205765 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:36.205775 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:36.206694 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:36.206697 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:36.206793 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:36.206822 (+ 29us) client_negotiation.cc:769] Sending connection context
0929 10:33:36.206896 (+ 74us) client_negotiation.cc:240] Negotiation successful
0929 10:33:36.206969 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77,"mutex_wait_us":36}
I20260929 10:33:36.207154 23130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:36.202557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33394 (local address 127.15.82.65:45833)
0929 10:33:36.203166 (+ 609us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:36.203171 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:36.203184 (+ 13us) server_negotiation.cc:407] Connection header received
0929 10:33:36.203236 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:36.203241 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:36.203291 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:36.203651 (+ 360us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:33:36.203769 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:36.204545 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:36.205903 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:36.206564 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:36.206685 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:36.206749 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:36.206924 (+ 175us) server_negotiation.cc:299] Negotiation successful
0929 10:33:36.207023 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":524,"thread_start_us":86,"threads_started":1}
I20260929 10:33:36.207311 23131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:36.202746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45644 (local address 127.15.82.67:43005)
0929 10:33:36.203012 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:36.203019 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:36.203032 (+ 13us) server_negotiation.cc:407] Connection header received
0929 10:33:36.203084 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:36.203087 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:36.203146 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:36.203540 (+ 394us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:33:36.203661 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:36.204448 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:36.205573 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:36.206226 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:36.207013 (+ 787us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:36.207074 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:36.207109 (+ 35us) server_negotiation.cc:299] Negotiation successful
0929 10:33:36.207178 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":71,"threads_started":1}
I20260929 10:33:36.207912 22976 tablet_service.cc:1467] Processing CreateTablet for tablet bf0f57f9d8ab47cdba4b1198886c7631 (DEFAULT_TABLE table=test-table [id=482add70d169432a9b7d05b05c32a88d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:33:36.208027 23132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:36.203407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.66:43097 (local address 127.0.0.1:41178)
0929 10:33:36.204049 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:36.204063 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:36.204154 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:36.204387 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:36.204390 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:36.204404 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:33:36.204495 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:36.204504 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:36.205526 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:36.205530 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:36.206732 (+ 1202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:36.206740 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:36.207640 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:36.207643 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:36.207731 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:36.207753 (+ 22us) client_negotiation.cc:769] Sending connection context
0929 10:33:36.207832 (+ 79us) client_negotiation.cc:240] Negotiation successful
0929 10:33:36.207894 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":586,"mutex_wait_us":47,"thread_start_us":97,"threads_started":1}
I20260929 10:33:36.208088 23133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:36.203759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41178 (local address 127.15.82.66:43097)
0929 10:33:36.204014 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:36.204019 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:36.204140 (+ 121us) server_negotiation.cc:407] Connection header received
0929 10:33:36.204232 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:36.204235 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:36.204281 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:36.204367 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:33:36.204633 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:36.205394 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:36.206833 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:36.207521 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:36.207634 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:36.207692 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:36.207867 (+ 175us) server_negotiation.cc:299] Negotiation successful
0929 10:33:36.207945 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":74,"threads_started":1}
I20260929 10:33:36.208263 22976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf0f57f9d8ab47cdba4b1198886c7631. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:33:36.208230 22702 tablet_service.cc:1467] Processing CreateTablet for tablet bf0f57f9d8ab47cdba4b1198886c7631 (DEFAULT_TABLE table=test-table [id=482add70d169432a9b7d05b05c32a88d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:33:36.208457 22702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf0f57f9d8ab47cdba4b1198886c7631. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:33:36.210302 23134 tablet_bootstrap.cc:492] T bf0f57f9d8ab47cdba4b1198886c7631 P d2e85165e3af490ea649ff60a3d44c65: Bootstrap starting.
I20260929 10:33:36.210587 23135 tablet_bootstrap.cc:492] T bf0f57f9d8ab47cdba4b1198886c7631 P 068171b1b6514121936f4634de5a8064: Bootstrap starting.
I20260929 10:33:36.211045 22839 tablet_service.cc:1467] Processing CreateTablet for tablet bf0f57f9d8ab47cdba4b1198886c7631 (DEFAULT_TABLE table=test-table [id=482add70d169432a9b7d05b05c32a88d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:33:36.211299 22839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf0f57f9d8ab47cdba4b1198886c7631. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:33:36.211354 23134 tablet_bootstrap.cc:654] T bf0f57f9d8ab47cdba4b1198886c7631 P d2e85165e3af490ea649ff60a3d44c65: Neither blocks nor log segments found. Creating new log.
I20260929 10:33:36.211390 23135 tablet_bootstrap.cc:654] T bf0f57f9d8ab47cdba4b1198886c7631 P 068171b1b6514121936f4634de5a8064: Neither blocks nor log segments found. Creating new log.
I20260929 10:33:36.212688 23134 tablet_bootstrap.cc:492] T bf0f57f9d8ab47cdba4b1198886c7631 P d2e85165e3af490ea649ff60a3d44c65: No bootstrap required, opened a new log
I20260929 10:33:36.212783 23134 ts_tablet_manager.cc:1397] T bf0f57f9d8ab47cdba4b1198886c7631 P d2e85165e3af490ea649ff60a3d44c65: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260929 10:33:36.213186 23134 raft_consensus.cc:348] T bf0f57f9d8ab47cdba4b1198886c7631 P d2e85165e3af490ea649ff60a3d44c65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068171b1b6514121936f4634de5a8064" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45833 } } peers { permanent_uuid: "d2e85165e3af490ea649ff60a3d44c65" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 43005 } } peers { permanent_uuid: "9886d2dce2354767a0ee1823c1021808" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43097 } }
I20260929 10:33:36.213310 23134 raft_consensus.cc:374] T bf0f57f9d8ab47cdba4b1198886c7631 P d2e85165e3af490ea649ff60a3d44c65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:33:36.213349 23134 raft_consensus.cc:729] T bf0f57f9d8ab47cdba4b1198886c7631 P d2e85165e3af490ea649ff60a3d44c65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2e85165e3af490ea649ff60a3d44c65, State: Initialized, Role: FOLLOWER
I20260929 10:33:36.213479 23134 consensus_queue.cc:260] T bf0f57f9d8ab47cdba4b1198886c7631 P d2e85165e3af490ea649ff60a3d44c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068171b1b6514121936f4634de5a8064" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45833 } } peers { permanent_uuid: "d2e85165e3af490ea649ff60a3d44c65" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 43005 } } peers { permanent_uuid: "9886d2dce2354767a0ee1823c1021808" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43097 } }
I20260929 10:33:36.213758 23138 tablet_bootstrap.cc:492] T bf0f57f9d8ab47cdba4b1198886c7631 P 9886d2dce2354767a0ee1823c1021808: Bootstrap starting.
I20260929 10:33:36.213853 23134 ts_tablet_manager.cc:1428] T bf0f57f9d8ab47cdba4b1198886c7631 P d2e85165e3af490ea649ff60a3d44c65: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260929 10:33:36.214485 23135 tablet_bootstrap.cc:492] T bf0f57f9d8ab47cdba4b1198886c7631 P 068171b1b6514121936f4634de5a8064: No bootstrap required, opened a new log
I20260929 10:33:36.214589 23135 ts_tablet_manager.cc:1397] T bf0f57f9d8ab47cdba4b1198886c7631 P 068171b1b6514121936f4634de5a8064: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260929 10:33:36.215080 23140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:36.214292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:42245 (local address 127.15.82.67:53489)
0929 10:33:36.214899 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:36.214961 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:42245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":134,"threads_started":1}
I20260929 10:33:36.215057 23135 raft_consensus.cc:348] T bf0f57f9d8ab47cdba4b1198886c7631 P 068171b1b6514121936f4634de5a8064 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068171b1b6514121936f4634de5a8064" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45833 } } peers { permanent_uuid: "d2e85165e3af490ea649ff60a3d44c65" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 43005 } } peers { permanent_uuid: "9886d2dce2354767a0ee1823c1021808" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43097 } }
I20260929 10:33:36.215175 23135 raft_consensus.cc:374] T bf0f57f9d8ab47cdba4b1198886c7631 P 068171b1b6514121936f4634de5a8064 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:33:36.215214 23135 raft_consensus.cc:729] T bf0f57f9d8ab47cdba4b1198886c7631 P 068171b1b6514121936f4634de5a8064 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 068171b1b6514121936f4634de5a8064, State: Initialized, Role: FOLLOWER
I20260929 10:33:36.215365 23135 consensus_queue.cc:260] T bf0f57f9d8ab47cdba4b1198886c7631 P 068171b1b6514121936f4634de5a8064 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068171b1b6514121936f4634de5a8064" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45833 } } peers { permanent_uuid: "d2e85165e3af490ea649ff60a3d44c65" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 43005 } } peers { permanent_uuid: "9886d2dce2354767a0ee1823c1021808" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43097 } }
I20260929 10:33:36.215737 23135 ts_tablet_manager.cc:1428] T bf0f57f9d8ab47cdba4b1198886c7631 P 068171b1b6514121936f4634de5a8064: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260929 10:33:36.215883 23142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:36.215490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:42245 (local address 127.15.82.67:43813)
0929 10:33:36.215710 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:36.215766 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:42245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":83,"threads_started":1}
I20260929 10:33:36.217692 23042 heartbeater.cc:503] Master 127.15.82.124:41423 was elected leader, sending a full tablet report...
I20260929 10:33:36.218768 23138 tablet_bootstrap.cc:654] T bf0f57f9d8ab47cdba4b1198886c7631 P 9886d2dce2354767a0ee1823c1021808: Neither blocks nor log segments found. Creating new log.
I20260929 10:33:36.219492 23143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:36.217258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:42245 (local address 127.15.82.65:41283)
0929 10:33:36.219364 (+ 2106us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:36.219410 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:42245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2048,"thread_start_us":97,"threads_started":1}
I20260929 10:33:36.220003 23138 tablet_bootstrap.cc:492] T bf0f57f9d8ab47cdba4b1198886c7631 P 9886d2dce2354767a0ee1823c1021808: No bootstrap required, opened a new log
I20260929 10:33:36.220084 23138 ts_tablet_manager.cc:1397] T bf0f57f9d8ab47cdba4b1198886c7631 P 9886d2dce2354767a0ee1823c1021808: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260929 10:33:36.220454 23138 raft_consensus.cc:348] T bf0f57f9d8ab47cdba4b1198886c7631 P 9886d2dce2354767a0ee1823c1021808 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068171b1b6514121936f4634de5a8064" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45833 } } peers { permanent_uuid: "d2e85165e3af490ea649ff60a3d44c65" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 43005 } } peers { permanent_uuid: "9886d2dce2354767a0ee1823c1021808" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43097 } }
I20260929 10:33:36.220546 23138 raft_consensus.cc:374] T bf0f57f9d8ab47cdba4b1198886c7631 P 9886d2dce2354767a0ee1823c1021808 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:33:36.220568 23138 raft_consensus.cc:729] T bf0f57f9d8ab47cdba4b1198886c7631 P 9886d2dce2354767a0ee1823c1021808 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9886d2dce2354767a0ee1823c1021808, State: Initialized, Role: FOLLOWER
I20260929 10:33:36.220662 23138 consensus_queue.cc:260] T bf0f57f9d8ab47cdba4b1198886c7631 P 9886d2dce2354767a0ee1823c1021808 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068171b1b6514121936f4634de5a8064" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45833 } } peers { permanent_uuid: "d2e85165e3af490ea649ff60a3d44c65" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 43005 } } peers { permanent_uuid: "9886d2dce2354767a0ee1823c1021808" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43097 } }
I20260929 10:33:36.220911 23138 ts_tablet_manager.cc:1428] T bf0f57f9d8ab47cdba4b1198886c7631 P 9886d2dce2354767a0ee1823c1021808: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260929 10:33:36.221805 23146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:36.221408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:42245 (local address 127.15.82.66:33349)
0929 10:33:36.221654 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:36.221712 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:42245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":103,"threads_started":1}
I20260929 10:33:36.222453 22905 heartbeater.cc:503] Master 127.15.82.124:41423 was elected leader, sending a full tablet report...
I20260929 10:33:36.222628 23146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:36.222262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:42245 (local address 127.15.82.66:49569)
0929 10:33:36.222517 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:36.222567 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:42245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180}
I20260929 10:33:36.235324 23139 raft_consensus.cc:482] T bf0f57f9d8ab47cdba4b1198886c7631 P d2e85165e3af490ea649ff60a3d44c65 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:33:36.235491 23139 raft_consensus.cc:504] T bf0f57f9d8ab47cdba4b1198886c7631 P d2e85165e3af490ea649ff60a3d44c65 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068171b1b6514121936f4634de5a8064" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45833 } } peers { permanent_uuid: "d2e85165e3af490ea649ff60a3d44c65" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 43005 } } peers { permanent_uuid: "9886d2dce2354767a0ee1823c1021808" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43097 } }
I20260929 10:33:36.236965 23139 leader_election.cc:288] T bf0f57f9d8ab47cdba4b1198886c7631 P d2e85165e3af490ea649ff60a3d44c65 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 068171b1b6514121936f4634de5a8064 (127.15.82.65:45833), 9886d2dce2354767a0ee1823c1021808 (127.15.82.66:43097)
I20260929 10:33:36.240959 23130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:36.237295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.67:47051 (local address 127.15.82.65:45833)
0929 10:33:36.237434 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:36.237440 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:36.237456 (+ 16us) server_negotiation.cc:407] Connection header received
0929 10:33:36.237625 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:36.237628 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:36.237675 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:36.237752 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:33:36.238096 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:36.238740 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:36.239869 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:36.240271 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:36.240412 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:36.240469 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:36.240775 (+ 306us) server_negotiation.cc:299] Negotiation successful
0929 10:33:36.240845 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260929 10:33:36.240979 23140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:36.237455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.66:43097 (local address 127.15.82.67:43233)
0929 10:33:36.237546 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:36.237580 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:36.237660 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:36.237918 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:36.237920 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:36.237933 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:33:36.238054 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:36.238062 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:36.238944 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:36.238950 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:36.239750 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:36.239759 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:36.240424 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:36.240427 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:36.240621 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:36.240647 (+ 26us) client_negotiation.cc:769] Sending connection context
0929 10:33:36.240710 (+ 63us) client_negotiation.cc:240] Negotiation successful
0929 10:33:36.240811 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26,"mutex_wait_us":53}
I20260929 10:33:36.240979 23133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:36.237577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.67:43233 (local address 127.15.82.66:43097)
0929 10:33:36.237685 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:36.237689 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:36.237738 (+ 49us) server_negotiation.cc:407] Connection header received
0929 10:33:36.237792 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:36.237794 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:36.237840 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:36.237910 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:33:36.238182 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:36.238795 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:36.239873 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:36.240271 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:36.240424 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:36.240476 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:36.240798 (+ 322us) server_negotiation.cc:299] Negotiation successful
0929 10:33:36.240871 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260929 10:33:36.240983 23142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:36.237184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.65:45833 (local address 127.15.82.67:47051)
0929 10:33:36.237320 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:36.237337 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:36.237464 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:36.237798 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:36.237801 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:36.237820 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:33:36.237977 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:36.237986 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:36.238841 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:36.238845 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:36.239747 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:36.239756 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:36.240430 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:36.240434 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:36.240623 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:36.240649 (+ 26us) client_negotiation.cc:769] Sending connection context
0929 10:33:36.240757 (+ 108us) client_negotiation.cc:240] Negotiation successful
0929 10:33:36.240842 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47,"mutex_wait_us":65}
I20260929 10:33:36.241482 22722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf0f57f9d8ab47cdba4b1198886c7631" candidate_uuid: "d2e85165e3af490ea649ff60a3d44c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "068171b1b6514121936f4634de5a8064" is_pre_election: true
I20260929 10:33:36.241642 22722 raft_consensus.cc:2454] T bf0f57f9d8ab47cdba4b1198886c7631 P 068171b1b6514121936f4634de5a8064 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2e85165e3af490ea649ff60a3d44c65 in term 0.
I20260929 10:33:36.241672 22859 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf0f57f9d8ab47cdba4b1198886c7631" candidate_uuid: "d2e85165e3af490ea649ff60a3d44c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9886d2dce2354767a0ee1823c1021808" is_pre_election: true
I20260929 10:33:36.241914 22859 raft_consensus.cc:2454] T bf0f57f9d8ab47cdba4b1198886c7631 P 9886d2dce2354767a0ee1823c1021808 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2e85165e3af490ea649ff60a3d44c65 in term 0.
I20260929 10:33:36.242017 22929 leader_election.cc:302] T bf0f57f9d8ab47cdba4b1198886c7631 P d2e85165e3af490ea649ff60a3d44c65 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068171b1b6514121936f4634de5a8064, d2e85165e3af490ea649ff60a3d44c65; no voters:
I20260929 10:33:36.242337 23139 raft_consensus.cc:2781] T bf0f57f9d8ab47cdba4b1198886c7631 P d2e85165e3af490ea649ff60a3d44c65 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:33:36.242434 23139 raft_consensus.cc:482] T bf0f57f9d8ab47cdba4b1198886c7631 P d2e85165e3af490ea649ff60a3d44c65 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:33:36.242491 23139 raft_consensus.cc:3037] T bf0f57f9d8ab47cdba4b1198886c7631 P d2e85165e3af490ea649ff60a3d44c65 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:33:36.243484 23139 raft_consensus.cc:504] T bf0f57f9d8ab47cdba4b1198886c7631 P d2e85165e3af490ea649ff60a3d44c65 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068171b1b6514121936f4634de5a8064" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45833 } } peers { permanent_uuid: "d2e85165e3af490ea649ff60a3d44c65" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 43005 } } peers { permanent_uuid: "9886d2dce2354767a0ee1823c1021808" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43097 } }
I20260929 10:33:36.243834 23139 leader_election.cc:288] T bf0f57f9d8ab47cdba4b1198886c7631 P d2e85165e3af490ea649ff60a3d44c65 [CANDIDATE]: Term 1 election: Requested vote from peers 068171b1b6514121936f4634de5a8064 (127.15.82.65:45833), 9886d2dce2354767a0ee1823c1021808 (127.15.82.66:43097)
I20260929 10:33:36.244233 22859 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf0f57f9d8ab47cdba4b1198886c7631" candidate_uuid: "d2e85165e3af490ea649ff60a3d44c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9886d2dce2354767a0ee1823c1021808"
I20260929 10:33:36.244241 22722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf0f57f9d8ab47cdba4b1198886c7631" candidate_uuid: "d2e85165e3af490ea649ff60a3d44c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "068171b1b6514121936f4634de5a8064"
I20260929 10:33:36.244369 22859 raft_consensus.cc:3037] T bf0f57f9d8ab47cdba4b1198886c7631 P 9886d2dce2354767a0ee1823c1021808 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:33:36.244374 22722 raft_consensus.cc:3037] T bf0f57f9d8ab47cdba4b1198886c7631 P 068171b1b6514121936f4634de5a8064 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:33:36.245333 22859 raft_consensus.cc:2454] T bf0f57f9d8ab47cdba4b1198886c7631 P 9886d2dce2354767a0ee1823c1021808 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2e85165e3af490ea649ff60a3d44c65 in term 1.
I20260929 10:33:36.245565 22722 raft_consensus.cc:2454] T bf0f57f9d8ab47cdba4b1198886c7631 P 068171b1b6514121936f4634de5a8064 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2e85165e3af490ea649ff60a3d44c65 in term 1.
I20260929 10:33:36.245627 22929 leader_election.cc:302] T bf0f57f9d8ab47cdba4b1198886c7631 P d2e85165e3af490ea649ff60a3d44c65 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9886d2dce2354767a0ee1823c1021808, d2e85165e3af490ea649ff60a3d44c65; no voters:
I20260929 10:33:36.245786 23139 raft_consensus.cc:2781] T bf0f57f9d8ab47cdba4b1198886c7631 P d2e85165e3af490ea649ff60a3d44c65 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:33:36.246017 23139 raft_consensus.cc:686] T bf0f57f9d8ab47cdba4b1198886c7631 P d2e85165e3af490ea649ff60a3d44c65 [term 1 LEADER]: Becoming Leader. State: Replica: d2e85165e3af490ea649ff60a3d44c65, State: Running, Role: LEADER
I20260929 10:33:36.246523 23139 consensus_queue.cc:237] T bf0f57f9d8ab47cdba4b1198886c7631 P d2e85165e3af490ea649ff60a3d44c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068171b1b6514121936f4634de5a8064" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45833 } } peers { permanent_uuid: "d2e85165e3af490ea649ff60a3d44c65" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 43005 } } peers { permanent_uuid: "9886d2dce2354767a0ee1823c1021808" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43097 } }
I20260929 10:33:36.246941 23142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:36.246674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:42245 (local address 127.15.82.67:38147)
0929 10:33:36.246819 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:36.246871 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:42245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260929 10:33:36.249109 22583 catalog_manager.cc:5495] T bf0f57f9d8ab47cdba4b1198886c7631 P d2e85165e3af490ea649ff60a3d44c65 reported cstate change: term changed from 0 to 1, leader changed from <none> to d2e85165e3af490ea649ff60a3d44c65 (127.15.82.67). New cstate: current_term: 1 leader_uuid: "d2e85165e3af490ea649ff60a3d44c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068171b1b6514121936f4634de5a8064" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45833 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2e85165e3af490ea649ff60a3d44c65" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 43005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9886d2dce2354767a0ee1823c1021808" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43097 } health_report { overall_health: UNKNOWN } } }
I20260929 10:33:36.265520 23131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:36.260447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45660 (local address 127.15.82.67:43005)
0929 10:33:36.260583 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:36.260588 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:36.260606 (+ 18us) server_negotiation.cc:407] Connection header received
0929 10:33:36.260695 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:36.260698 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:36.260750 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:36.260856 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:33:36.261101 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:36.261901 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:36.263508 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:36.263865 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:36.263988 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:36.264291 (+ 303us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:33:36.265038 (+ 747us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:33:36.265187 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:36.265327 (+ 140us) server_negotiation.cc:299] Negotiation successful
0929 10:33:36.265383 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260929 10:33:36.268841 22722 raft_consensus.cc:1261] T bf0f57f9d8ab47cdba4b1198886c7631 P 068171b1b6514121936f4634de5a8064 [term 1 FOLLOWER]: Refusing update from remote peer d2e85165e3af490ea649ff60a3d44c65: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:33:36.269102 22859 raft_consensus.cc:1261] T bf0f57f9d8ab47cdba4b1198886c7631 P 9886d2dce2354767a0ee1823c1021808 [term 1 FOLLOWER]: Refusing update from remote peer d2e85165e3af490ea649ff60a3d44c65: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:33:36.269644 23139 consensus_queue.cc:1035] T bf0f57f9d8ab47cdba4b1198886c7631 P d2e85165e3af490ea649ff60a3d44c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "068171b1b6514121936f4634de5a8064" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 45833 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:33:36.269860 23147 consensus_queue.cc:1035] T bf0f57f9d8ab47cdba4b1198886c7631 P d2e85165e3af490ea649ff60a3d44c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9886d2dce2354767a0ee1823c1021808" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 43097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:33:36.271730 22768 heartbeater.cc:503] Master 127.15.82.124:41423 was elected leader, sending a full tablet report...
I20260929 10:33:36.272364 23146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:36.272050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:42245 (local address 127.15.82.66:40643)
0929 10:33:36.272219 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:36.272284 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:42245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260929 10:33:36.272847 23143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:36.272506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:42245 (local address 127.15.82.65:41189)
0929 10:33:36.272728 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:36.272785 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:42245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152}
I20260929 10:33:36.276618 23143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:36.276382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:42245 (local address 127.15.82.65:48047)
0929 10:33:36.276480 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:36.276547 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:42245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260929 10:33:36.276911 23142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:36.276382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:42245 (local address 127.15.82.67:55535)
0929 10:33:36.276784 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:36.276844 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:42245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":334}
I20260929 10:33:36.277869 23146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:36.277618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:42245 (local address 127.15.82.66:58823)
0929 10:33:36.277755 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:36.277804 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:42245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260929 10:33:36.279361 23142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:36.279142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:42245 (local address 127.15.82.67:50979)
0929 10:33:36.279226 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:36.279288 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:42245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260929 10:33:36.290730 22586 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38116:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:33:36.290977 22586 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38116:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:33:36.295689 22586 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9df9b9ceadd74c56bc5c0242855d0777: Sending DeleteTablet for 3 replicas of tablet bf0f57f9d8ab47cdba4b1198886c7631
I20260929 10:33:36.296408 22702 tablet_service.cc:1514] Processing DeleteTablet for tablet bf0f57f9d8ab47cdba4b1198886c7631 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:33:36 UTC) from {username='oryx', principal='oryx/127.15.82.124@KRBTEST.COM'} at 127.0.0.1:33394
I20260929 10:33:36.296761 22839 tablet_service.cc:1514] Processing DeleteTablet for tablet bf0f57f9d8ab47cdba4b1198886c7631 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:33:36 UTC) from {username='oryx', principal='oryx/127.15.82.124@KRBTEST.COM'} at 127.0.0.1:41178
I20260929 10:33:36.297037 23161 tablet_replica.cc:330] T bf0f57f9d8ab47cdba4b1198886c7631 P 068171b1b6514121936f4634de5a8064: stopping tablet replica
I20260929 10:33:36.297216 23161 raft_consensus.cc:2229] T bf0f57f9d8ab47cdba4b1198886c7631 P 068171b1b6514121936f4634de5a8064 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:33:36.297264 22976 tablet_service.cc:1514] Processing DeleteTablet for tablet bf0f57f9d8ab47cdba4b1198886c7631 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:33:36 UTC) from {username='oryx', principal='oryx/127.15.82.124@KRBTEST.COM'} at 127.0.0.1:45644
I20260929 10:33:36.297497 23161 raft_consensus.cc:2258] T bf0f57f9d8ab47cdba4b1198886c7631 P 068171b1b6514121936f4634de5a8064 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:33:36.297971 23162 tablet_replica.cc:330] T bf0f57f9d8ab47cdba4b1198886c7631 P 9886d2dce2354767a0ee1823c1021808: stopping tablet replica
I20260929 10:33:36.298091 23162 raft_consensus.cc:2229] T bf0f57f9d8ab47cdba4b1198886c7631 P 9886d2dce2354767a0ee1823c1021808 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:33:36.298221 23162 raft_consensus.cc:2258] T bf0f57f9d8ab47cdba4b1198886c7631 P 9886d2dce2354767a0ee1823c1021808 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:33:36.298501 23163 tablet_replica.cc:330] T bf0f57f9d8ab47cdba4b1198886c7631 P d2e85165e3af490ea649ff60a3d44c65: stopping tablet replica
I20260929 10:33:36.298661 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 22551
I20260929 10:33:36.299723 23163 raft_consensus.cc:2229] T bf0f57f9d8ab47cdba4b1198886c7631 P d2e85165e3af490ea649ff60a3d44c65 [term 1 LEADER]: Raft consensus shutting down.
I20260929 10:33:36.300720 23162 ts_tablet_manager.cc:1905] T bf0f57f9d8ab47cdba4b1198886c7631 P 9886d2dce2354767a0ee1823c1021808: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:33:36.304109 23162 ts_tablet_manager.cc:1918] T bf0f57f9d8ab47cdba4b1198886c7631 P 9886d2dce2354767a0ee1823c1021808: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:33:36.304203 23162 log.cc:1196] T bf0f57f9d8ab47cdba4b1198886c7631 P 9886d2dce2354767a0ee1823c1021808: Deleting WAL directory at /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-1/wal/wals/bf0f57f9d8ab47cdba4b1198886c7631
I20260929 10:33:36.304492 23162 ts_tablet_manager.cc:1939] T bf0f57f9d8ab47cdba4b1198886c7631 P 9886d2dce2354767a0ee1823c1021808: Deleting consensus metadata
I20260929 10:33:36.305359 22573 catalog_manager.cc:4841] TS 9886d2dce2354767a0ee1823c1021808 (127.15.82.66:43097): tablet bf0f57f9d8ab47cdba4b1198886c7631 (table test-table [id=482add70d169432a9b7d05b05c32a88d]) successfully deleted
I20260929 10:33:36.305852 23163 raft_consensus.cc:2258] T bf0f57f9d8ab47cdba4b1198886c7631 P d2e85165e3af490ea649ff60a3d44c65 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:33:36.307340 23163 ts_tablet_manager.cc:1905] T bf0f57f9d8ab47cdba4b1198886c7631 P d2e85165e3af490ea649ff60a3d44c65: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:33:36.309484 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 22776
W20260929 10:33:36.309522 23132 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:33:36.309118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.65:45833 (local address 127.0.0.1:33404)
0929 10:33:36.309325 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:36.309393 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.65:45833: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":109}
W20260929 10:33:36.310387 22573 catalog_manager.cc:4584] TS 068171b1b6514121936f4634de5a8064 (127.15.82.65:45833): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bf0f57f9d8ab47cdba4b1198886c7631: Network error: Client connection negotiation failed: client connection to 127.15.82.65:45833: connect: Connection reset by peer (error 104)
I20260929 10:33:36.312865 23163 ts_tablet_manager.cc:1918] T bf0f57f9d8ab47cdba4b1198886c7631 P d2e85165e3af490ea649ff60a3d44c65: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:33:36.312961 23163 log.cc:1196] T bf0f57f9d8ab47cdba4b1198886c7631 P d2e85165e3af490ea649ff60a3d44c65: Deleting WAL directory at /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790677981592896-15689-0/minicluster-data/ts-2/wal/wals/bf0f57f9d8ab47cdba4b1198886c7631
I20260929 10:33:36.313251 23163 ts_tablet_manager.cc:1939] T bf0f57f9d8ab47cdba4b1198886c7631 P d2e85165e3af490ea649ff60a3d44c65: Deleting consensus metadata
I20260929 10:33:36.314101 22575 catalog_manager.cc:4841] TS d2e85165e3af490ea649ff60a3d44c65 (127.15.82.67:43005): tablet bf0f57f9d8ab47cdba4b1198886c7631 (table test-table [id=482add70d169432a9b7d05b05c32a88d]) successfully deleted
I20260929 10:33:36.319990 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 22912
I20260929 10:33:36.325325 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 22268
I20260929 10:33:36.331323 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 22378
2026-09-29T10:33:36Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (5314 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:33:36 dist-test-slave-vg89 krb5kdc[23166](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:33:36 dist-test-slave-vg89 krb5kdc[23166](info): set up 2 sockets
Sep 29 10:33:36 dist-test-slave-vg89 krb5kdc[23166](info): commencing operation
krb5kdc: starting...
W20260929 10:33:37.371436 15689 mini_kdc.cc:121] Time spent starting KDC: real 1.016s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Sep 29 10:33:37 dist-test-slave-vg89 krb5kdc[23166](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678017, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:33:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:33:37Z Disabled control of system clock
WARNING: no policy specified for oryx/127.15.82.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.82.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.82.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/krb5kdc/oryx_127.15.82.126.keytab.
Entry for principal oryx/127.15.82.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/krb5kdc/oryx_127.15.82.126.keytab.
I20260929 10:33:37.505971 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:41463
--webserver_interface=127.15.82.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/krb5kdc/oryx_127.15.82.126.keytab
--principal=oryx/127.15.82.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:43047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.82.126:41463
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:33:37.626746 23179 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:33:37.627031 23179 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:33:37.627110 23179 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:33:37.631167 23179 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:33:37.631232 23179 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260929 10:33:37.631248 23179 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:33:37.631266 23179 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:33:37.631284 23179 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:33:37.631299 23179 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:33:37.636641 23179 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:43047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.82.126:41463
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/krb5kdc/oryx_127.15.82.126.keytab
--principal=oryx/127.15.82.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:41463
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.82.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23179
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:33:37.637733 23179 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:33:37.638842 23179 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 10:33:37.647603 23179 server_base.cc:1034] running on GCE node
W20260929 10:33:37.647533 23185 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:37.647528 23187 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:37.647533 23184 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:37.648476 23179 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:33:37.649544 23179 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:33:37.650704 23179 hybrid_clock.cc:648] HybridClock initialized: now 1790678017650691 us; error 25 us; skew 500 ppm
Sep 29 10:33:37 dist-test-slave-vg89 krb5kdc[23166](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678017, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.82.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:33:37.653559 23179 init.cc:377] Logged in from keytab as oryx/127.15.82.126@KRBTEST.COM (short username oryx)
I20260929 10:33:37.654502 23179 webserver.cc:460] Webserver started at http://127.15.82.126:33837/ using document root <none> and password file <none>
I20260929 10:33:37.655071 23179 fs_manager.cc:359] Metadata directory not provided
I20260929 10:33:37.655164 23179 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:33:37.655406 23179 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:33:37.657460 23179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/master-0/data/instance:
uuid: "f5a8c72fbf5a49f0a0dbf4031fe3ff93"
format_stamp: "Formatted at 2026-09-29 10:33:37 on dist-test-slave-vg89"
server_key: "1a5e8e983c5c42cd74f23f370ce1afef"
server_key_iv: "18c6b8deddc13dfe621dd2ab16c1a7e1"
server_key_version: "encryptionkey@0"
I20260929 10:33:37.658054 23179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/master-0/wal/instance:
uuid: "f5a8c72fbf5a49f0a0dbf4031fe3ff93"
format_stamp: "Formatted at 2026-09-29 10:33:37 on dist-test-slave-vg89"
server_key: "1a5e8e983c5c42cd74f23f370ce1afef"
server_key_iv: "18c6b8deddc13dfe621dd2ab16c1a7e1"
server_key_version: "encryptionkey@0"
I20260929 10:33:37.661835 23179 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260929 10:33:37.664305 23194 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:37.665542 23179 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 10:33:37.665697 23179 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/master-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/master-0/wal
uuid: "f5a8c72fbf5a49f0a0dbf4031fe3ff93"
format_stamp: "Formatted at 2026-09-29 10:33:37 on dist-test-slave-vg89"
server_key: "1a5e8e983c5c42cd74f23f370ce1afef"
server_key_iv: "18c6b8deddc13dfe621dd2ab16c1a7e1"
server_key_version: "encryptionkey@0"
I20260929 10:33:37.665817 23179 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:33:37.684616 23179 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:33:37.688210 23179 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:33:37.688437 23179 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:33:37.696777 23179 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.126:41463
I20260929 10:33:37.696791 23256 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.126:41463 every 8 connection(s)
I20260929 10:33:37.697762 23179 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
I20260929 10:33:37.700896 23257 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:33:37.703176 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 23179
I20260929 10:33:37.703317 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/master-0/wal/instance
I20260929 10:33:37.703683 15689 external_mini_cluster.cc:1428] Setting key 3074a4b2167668e75ed8151d26cb85c5
I20260929 10:33:37.708380 23257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5a8c72fbf5a49f0a0dbf4031fe3ff93: Bootstrap starting.
I20260929 10:33:37.710860 23257 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f5a8c72fbf5a49f0a0dbf4031fe3ff93: Neither blocks nor log segments found. Creating new log.
I20260929 10:33:37.711644 23257 log.cc:824] T 00000000000000000000000000000000 P f5a8c72fbf5a49f0a0dbf4031fe3ff93: Log is configured to *not* fsync() on all Append() calls
Sep 29 10:33:37 dist-test-slave-vg89 krb5kdc[23166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678017, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM
I20260929 10:33:37.713864 23257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5a8c72fbf5a49f0a0dbf4031fe3ff93: No bootstrap required, opened a new log
I20260929 10:33:37.718547 23257 raft_consensus.cc:348] T 00000000000000000000000000000000 P f5a8c72fbf5a49f0a0dbf4031fe3ff93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5a8c72fbf5a49f0a0dbf4031fe3ff93" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 41463 } }
I20260929 10:33:37.718822 23257 raft_consensus.cc:374] T 00000000000000000000000000000000 P f5a8c72fbf5a49f0a0dbf4031fe3ff93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:33:37.718848 23260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:37.705273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35994 (local address 127.15.82.126:41463)
0929 10:33:37.705757 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:37.705773 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:37.705808 (+ 35us) server_negotiation.cc:407] Connection header received
0929 10:33:37.706647 (+ 839us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:37.706667 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:37.707006 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:37.707367 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:37.708188 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:37.709259 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:37.710111 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:37.710656 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:37.710831 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:37.713516 (+ 2685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:37.713523 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:37.713539 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:37.713570 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:37.715677 (+ 2107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:37.716156 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:37.716162 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:37.716168 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:37.716308 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:37.716583 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:37.716586 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:37.716588 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:37.717094 (+ 506us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:37.717291 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:37.717506 (+ 215us) server_negotiation.cc:299] Negotiation successful
0929 10:33:37.717831 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":159,"threads_started":1}
I20260929 10:33:37.718890 23257 raft_consensus.cc:729] T 00000000000000000000000000000000 P f5a8c72fbf5a49f0a0dbf4031fe3ff93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5a8c72fbf5a49f0a0dbf4031fe3ff93, State: Initialized, Role: FOLLOWER
I20260929 10:33:37.719380 23257 consensus_queue.cc:260] T 00000000000000000000000000000000 P f5a8c72fbf5a49f0a0dbf4031fe3ff93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5a8c72fbf5a49f0a0dbf4031fe3ff93" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 41463 } }
I20260929 10:33:37.719542 23257 raft_consensus.cc:388] T 00000000000000000000000000000000 P f5a8c72fbf5a49f0a0dbf4031fe3ff93 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:33:37.719604 23257 raft_consensus.cc:482] T 00000000000000000000000000000000 P f5a8c72fbf5a49f0a0dbf4031fe3ff93 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:33:37.719712 23257 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f5a8c72fbf5a49f0a0dbf4031fe3ff93 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:33:37.720649 23257 raft_consensus.cc:504] T 00000000000000000000000000000000 P f5a8c72fbf5a49f0a0dbf4031fe3ff93 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5a8c72fbf5a49f0a0dbf4031fe3ff93" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 41463 } }
I20260929 10:33:37.721004 23257 leader_election.cc:302] T 00000000000000000000000000000000 P f5a8c72fbf5a49f0a0dbf4031fe3ff93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f5a8c72fbf5a49f0a0dbf4031fe3ff93; no voters:
I20260929 10:33:37.721328 23257 leader_election.cc:288] T 00000000000000000000000000000000 P f5a8c72fbf5a49f0a0dbf4031fe3ff93 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:33:37.721446 23262 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f5a8c72fbf5a49f0a0dbf4031fe3ff93 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:33:37.721698 23262 raft_consensus.cc:686] T 00000000000000000000000000000000 P f5a8c72fbf5a49f0a0dbf4031fe3ff93 [term 1 LEADER]: Becoming Leader. State: Replica: f5a8c72fbf5a49f0a0dbf4031fe3ff93, State: Running, Role: LEADER
I20260929 10:33:37.722029 23262 consensus_queue.cc:237] T 00000000000000000000000000000000 P f5a8c72fbf5a49f0a0dbf4031fe3ff93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5a8c72fbf5a49f0a0dbf4031fe3ff93" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 41463 } }
I20260929 10:33:37.722692 23257 sys_catalog.cc:564] T 00000000000000000000000000000000 P f5a8c72fbf5a49f0a0dbf4031fe3ff93 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:33:37.723793 23263 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5a8c72fbf5a49f0a0dbf4031fe3ff93 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f5a8c72fbf5a49f0a0dbf4031fe3ff93" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5a8c72fbf5a49f0a0dbf4031fe3ff93" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 41463 } } }
I20260929 10:33:37.723920 23263 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5a8c72fbf5a49f0a0dbf4031fe3ff93 [sys.catalog]: This master's current role is: LEADER
I20260929 10:33:37.724215 23264 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5a8c72fbf5a49f0a0dbf4031fe3ff93 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f5a8c72fbf5a49f0a0dbf4031fe3ff93. Latest consensus state: current_term: 1 leader_uuid: "f5a8c72fbf5a49f0a0dbf4031fe3ff93" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5a8c72fbf5a49f0a0dbf4031fe3ff93" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 41463 } } }
I20260929 10:33:37.724308 23264 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5a8c72fbf5a49f0a0dbf4031fe3ff93 [sys.catalog]: This master's current role is: LEADER
I20260929 10:33:37.724493 23271 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:33:37.727777 23271 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:33:37.733657 23271 catalog_manager.cc:1347] Generated new cluster ID: cba1a8598e754ddfbffe7e7a573b1f23
I20260929 10:33:37.733738 23271 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:33:37.742607 23271 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:33:37.743577 23271 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:33:37.748631 23271 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f5a8c72fbf5a49f0a0dbf4031fe3ff93: Generated new TSK 0
I20260929 10:33:37.749328 23271 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.15.82.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.82.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.82.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/krb5kdc/oryx_127.15.82.65.keytab.
Entry for principal oryx/127.15.82.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/krb5kdc/oryx_127.15.82.65.keytab.
I20260929 10:33:37.785667 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.65:0
--local_ip_for_outbound_sockets=127.15.82.65
--webserver_interface=127.15.82.65
--webserver_port=0
--tserver_master_addrs=127.15.82.126:41463
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/krb5kdc/oryx_127.15.82.65.keytab
--principal=oryx/127.15.82.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:43047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:33:37.900008 23283 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:33:37.900337 23283 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:33:37.900419 23283 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:33:37.904470 23283 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:33:37.904572 23283 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:33:37.904659 23283 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 10:33:37.904739 23283 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.65
I20260929 10:33:37.910254 23283 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:43047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/krb5kdc/oryx_127.15.82.65.keytab
--principal=oryx/127.15.82.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.82.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.82.126:41463
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23283
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.65
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:33:37.911396 23283 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:33:37.912459 23283 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:33:37.922732 23289 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:37.922871 23283 server_base.cc:1034] running on GCE node
W20260929 10:33:37.922740 23288 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:37.922778 23291 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:37.923563 23283 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:33:37.924336 23283 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:33:37.925522 23283 hybrid_clock.cc:648] HybridClock initialized: now 1790678017925502 us; error 32 us; skew 500 ppm
Sep 29 10:33:37 dist-test-slave-vg89 krb5kdc[23166](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678017, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.82.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:33:37.928395 23283 init.cc:377] Logged in from keytab as oryx/127.15.82.65@KRBTEST.COM (short username oryx)
I20260929 10:33:37.929286 23283 webserver.cc:460] Webserver started at http://127.15.82.65:33579/ using document root <none> and password file <none>
I20260929 10:33:37.929919 23283 fs_manager.cc:359] Metadata directory not provided
I20260929 10:33:37.930044 23283 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:33:37.930282 23283 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:33:37.932453 23283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/ts-0/data/instance:
uuid: "e44ce81184b94969854575ed79d06c48"
format_stamp: "Formatted at 2026-09-29 10:33:37 on dist-test-slave-vg89"
server_key: "8d9e47bc9c9eb498bb9483c9704d6622"
server_key_iv: "d8dc55c1d877ae3fb04d6f07c02b9427"
server_key_version: "encryptionkey@0"
I20260929 10:33:37.933053 23283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/ts-0/wal/instance:
uuid: "e44ce81184b94969854575ed79d06c48"
format_stamp: "Formatted at 2026-09-29 10:33:37 on dist-test-slave-vg89"
server_key: "8d9e47bc9c9eb498bb9483c9704d6622"
server_key_iv: "d8dc55c1d877ae3fb04d6f07c02b9427"
server_key_version: "encryptionkey@0"
I20260929 10:33:37.936825 23283 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260929 10:33:37.939330 23298 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:37.940780 23283 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 10:33:37.940913 23283 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/ts-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/ts-0/wal
uuid: "e44ce81184b94969854575ed79d06c48"
format_stamp: "Formatted at 2026-09-29 10:33:37 on dist-test-slave-vg89"
server_key: "8d9e47bc9c9eb498bb9483c9704d6622"
server_key_iv: "d8dc55c1d877ae3fb04d6f07c02b9427"
server_key_version: "encryptionkey@0"
I20260929 10:33:37.941036 23283 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:33:37.953969 23283 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:33:37.957783 23283 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:33:37.958046 23283 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:33:37.959748 23283 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:33:37.959836 23283 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:37.959918 23283 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:33:37.959970 23283 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 10:33:37 dist-test-slave-vg89 krb5kdc[23166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678017, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.82.65@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM
I20260929 10:33:37.972558 23283 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.65:34763
I20260929 10:33:37.972640 23413 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.65:34763 every 8 connection(s)
I20260929 10:33:37.973748 23283 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
I20260929 10:33:37.975215 23260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:37.960659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.65:52409 (local address 127.15.82.126:41463)
0929 10:33:37.960791 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:37.960795 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:37.961907 (+ 1112us) server_negotiation.cc:407] Connection header received
0929 10:33:37.963085 (+ 1178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:37.963089 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:37.963176 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:37.963294 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:37.965105 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:37.966037 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:37.967630 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:37.967937 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:37.968062 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:37.970980 (+ 2918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:37.970984 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:37.970989 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:37.971023 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:37.972677 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:37.973327 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:37.973330 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:37.973331 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:37.973406 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:37.973826 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:37.973830 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:37.973831 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:37.974044 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:37.974146 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:37.974926 (+ 780us) server_negotiation.cc:299] Negotiation successful
0929 10:33:37.975053 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260929 10:33:37.976001 23307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:37.961250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:41463 (local address 127.15.82.65:52409)
0929 10:33:37.961749 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:37.961793 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:37.962808 (+ 1015us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:37.963630 (+ 822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:37.963637 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:37.964197 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:37.964896 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:37.964917 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:37.966210 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:37.966214 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:37.967488 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:37.967497 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:37.968134 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:37.968138 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:37.968313 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:37.968995 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:37.969018 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:37.970797 (+ 1779us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:37.972807 (+ 2010us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:37.972813 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:37.972827 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:37.973210 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:37.973514 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:37.973518 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:37.973520 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:37.973709 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:37.974268 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:37.974278 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:37.974668 (+ 390us) client_negotiation.cc:769] Sending connection context
0929 10:33:37.974867 (+ 199us) client_negotiation.cc:240] Negotiation successful
0929 10:33:37.975208 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":138,"threads_started":1}
I20260929 10:33:37.977456 23414 heartbeater.cc:347] Connected to a master server at 127.15.82.126:41463
I20260929 10:33:37.977626 23414 heartbeater.cc:464] Registering TS with master...
I20260929 10:33:37.978197 23414 heartbeater.cc:511] Master 127.15.82.126:41463 requested a full tablet report, sending...
I20260929 10:33:37.979897 23210 ts_manager.cc:194] Registered new tserver with Master: e44ce81184b94969854575ed79d06c48 (127.15.82.65:34763)
I20260929 10:33:37.981201 23210 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.82.65@KRBTEST.COM'} at 127.15.82.65:52409
I20260929 10:33:37.983346 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 23283
I20260929 10:33:37.983489 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/ts-0/wal/instance
I20260929 10:33:37.983836 15689 external_mini_cluster.cc:1428] Setting key a7b46d96b6b49eb291bea9e35a674c08
I20260929 10:33:37.987042 23260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:37.979642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.65:55549 (local address 127.15.82.126:41463)
0929 10:33:37.979789 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:37.979793 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:37.979939 (+ 146us) server_negotiation.cc:407] Connection header received
0929 10:33:37.980081 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:37.980084 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:37.980144 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:37.980211 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:37.980711 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:37.981213 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:37.982223 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:37.982488 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:37.982570 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:37.983881 (+ 1311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:37.983886 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:37.983889 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:37.983916 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:37.985416 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:37.985858 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:37.985861 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:37.985863 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:37.985943 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:37.986217 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:37.986221 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:37.986223 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:37.986389 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:37.986530 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:37.986777 (+ 247us) server_negotiation.cc:299] Negotiation successful
0929 10:33:37.986883 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260929 10:33:37.987195 23420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:37.979522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:41463 (local address 127.15.82.65:55549)
0929 10:33:37.979847 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:37.979862 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:37.979984 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:37.980254 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:37.980257 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:37.980472 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:37.980601 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:37.980608 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:37.981344 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:37.981347 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:37.982105 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:37.982117 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:37.982595 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:37.982599 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:37.982720 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:37.983283 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:37.983297 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:37.983725 (+ 428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:37.985545 (+ 1820us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:37.985548 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:37.985550 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:37.985754 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:37.986033 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:37.986037 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:37.986039 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:37.986127 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:37.986518 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:37.986521 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:37.986620 (+ 99us) client_negotiation.cc:769] Sending connection context
0929 10:33:37.986727 (+ 107us) client_negotiation.cc:240] Negotiation successful
0929 10:33:37.986872 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"mutex_wait_us":3,"thread_start_us":119,"threads_started":1}
WARNING: no policy specified for oryx/127.15.82.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.82.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.82.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/krb5kdc/oryx_127.15.82.66.keytab.
Entry for principal oryx/127.15.82.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/krb5kdc/oryx_127.15.82.66.keytab.
I20260929 10:33:38.013046 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.66:0
--local_ip_for_outbound_sockets=127.15.82.66
--webserver_interface=127.15.82.66
--webserver_port=0
--tserver_master_addrs=127.15.82.126:41463
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/krb5kdc/oryx_127.15.82.66.keytab
--principal=oryx/127.15.82.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:43047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:33:38.130328 23423 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:33:38.130695 23423 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:33:38.130779 23423 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:33:38.134876 23423 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:33:38.134963 23423 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:33:38.135036 23423 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 10:33:38.135116 23423 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.66
I20260929 10:33:38.140684 23423 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:43047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/krb5kdc/oryx_127.15.82.66.keytab
--principal=oryx/127.15.82.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.82.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.82.126:41463
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23423
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.66
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:33:38.141860 23423 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:33:38.143009 23423 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:33:38.153228 23428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:38.153374 23423 server_base.cc:1034] running on GCE node
W20260929 10:33:38.153244 23431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:38.153250 23429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:38.153904 23423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:33:38.154631 23423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:33:38.155820 23423 hybrid_clock.cc:648] HybridClock initialized: now 1790678018155806 us; error 30 us; skew 500 ppm
Sep 29 10:33:38 dist-test-slave-vg89 krb5kdc[23166](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678018, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.82.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:33:38.158629 23423 init.cc:377] Logged in from keytab as oryx/127.15.82.66@KRBTEST.COM (short username oryx)
I20260929 10:33:38.159554 23423 webserver.cc:460] Webserver started at http://127.15.82.66:34895/ using document root <none> and password file <none>
I20260929 10:33:38.160182 23423 fs_manager.cc:359] Metadata directory not provided
I20260929 10:33:38.160276 23423 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:33:38.160514 23423 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:33:38.162678 23423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/ts-1/data/instance:
uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7"
format_stamp: "Formatted at 2026-09-29 10:33:38 on dist-test-slave-vg89"
server_key: "e4c4c99f4b399e3b82d46d24633a562b"
server_key_iv: "a6ff32fc2dabb4ce1f09dab5d6b5c1b3"
server_key_version: "encryptionkey@0"
I20260929 10:33:38.163264 23423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/ts-1/wal/instance:
uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7"
format_stamp: "Formatted at 2026-09-29 10:33:38 on dist-test-slave-vg89"
server_key: "e4c4c99f4b399e3b82d46d24633a562b"
server_key_iv: "a6ff32fc2dabb4ce1f09dab5d6b5c1b3"
server_key_version: "encryptionkey@0"
I20260929 10:33:38.167591 23423 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 10:33:38.170492 23438 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:38.171870 23423 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 10:33:38.172014 23423 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/ts-1/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/ts-1/wal
uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7"
format_stamp: "Formatted at 2026-09-29 10:33:38 on dist-test-slave-vg89"
server_key: "e4c4c99f4b399e3b82d46d24633a562b"
server_key_iv: "a6ff32fc2dabb4ce1f09dab5d6b5c1b3"
server_key_version: "encryptionkey@0"
I20260929 10:33:38.172147 23423 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:33:38.192422 23423 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:33:38.196049 23423 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:33:38.196280 23423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:33:38.197811 23423 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:33:38.197890 23423 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:38.197962 23423 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:33:38.197993 23423 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 10:33:38 dist-test-slave-vg89 krb5kdc[23166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678018, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.82.66@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM
I20260929 10:33:38.209093 23423 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.66:34377
I20260929 10:33:38.209142 23553 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.66:34377 every 8 connection(s)
I20260929 10:33:38.210361 23423 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
I20260929 10:33:38.210712 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 23423
I20260929 10:33:38.210865 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/ts-1/wal/instance
I20260929 10:33:38.211244 15689 external_mini_cluster.cc:1428] Setting key ceeee3b56113b411a8fe470e49107c01
I20260929 10:33:38.212891 23260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:38.198786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:53215 (local address 127.15.82.126:41463)
0929 10:33:38.198921 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:38.198925 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:38.200082 (+ 1157us) server_negotiation.cc:407] Connection header received
0929 10:33:38.201276 (+ 1194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:38.201279 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:38.201340 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:38.201441 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:38.203238 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:38.204162 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:38.205371 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:38.205779 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:38.205889 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:38.208624 (+ 2735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:38.208628 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:38.208631 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:38.208665 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:38.210539 (+ 1874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:38.211201 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:38.211205 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:38.211206 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:38.211299 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:38.211688 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:38.211693 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:38.211696 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:38.211966 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:38.212077 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:38.212587 (+ 510us) server_negotiation.cc:299] Negotiation successful
0929 10:33:38.212748 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260929 10:33:38.214017 23447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:38.199391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:41463 (local address 127.15.82.66:53215)
0929 10:33:38.199924 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:38.199969 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:38.201044 (+ 1075us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:38.201670 (+ 626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:38.201682 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:38.202190 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:38.203012 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:38.203039 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:38.204287 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:38.204290 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:38.205257 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:38.205269 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:38.205950 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:38.205953 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:38.206114 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:38.206842 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:38.206860 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:38.208414 (+ 1554us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:38.210684 (+ 2270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:38.210690 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:38.210705 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:38.211068 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:38.211401 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:38.211404 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:38.211406 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:38.211595 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:38.212084 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:38.212090 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:38.212316 (+ 226us) client_negotiation.cc:769] Sending connection context
0929 10:33:38.212534 (+ 218us) client_negotiation.cc:240] Negotiation successful
0929 10:33:38.212916 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":163,"threads_started":1}
I20260929 10:33:38.215373 23554 heartbeater.cc:347] Connected to a master server at 127.15.82.126:41463
I20260929 10:33:38.215513 23554 heartbeater.cc:464] Registering TS with master...
I20260929 10:33:38.216073 23554 heartbeater.cc:511] Master 127.15.82.126:41463 requested a full tablet report, sending...
I20260929 10:33:38.217152 23211 ts_manager.cc:194] Registered new tserver with Master: c1f0b22653cc422d9b26cb3f6b39b1d7 (127.15.82.66:34377)
I20260929 10:33:38.217952 23211 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.82.66@KRBTEST.COM'} at 127.15.82.66:53215
WARNING: no policy specified for oryx/127.15.82.67@KRBTEST.COM; defaulting to no policy
I20260929 10:33:38.226712 23561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:38.217781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:41463 (local address 127.15.82.66:38231)
0929 10:33:38.218589 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:38.218606 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:38.218706 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:38.218943 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:38.218947 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:38.219174 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:38.219284 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:38.219292 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:38.220042 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:38.220045 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:38.221023 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:38.221039 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:38.221476 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:38.221482 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:38.221603 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:38.222169 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:38.222189 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:38.222749 (+ 560us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:38.224935 (+ 2186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:38.224938 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:38.224940 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:38.225192 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:38.225535 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:38.225537 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:38.225539 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:38.225624 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:38.226187 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:38.226190 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:38.226279 (+ 89us) client_negotiation.cc:769] Sending connection context
0929 10:33:38.226368 (+ 89us) client_negotiation.cc:240] Negotiation successful
0929 10:33:38.226559 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":721,"thread_start_us":90,"threads_started":1}
I20260929 10:33:38.226704 23260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:38.217861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:38231 (local address 127.15.82.126:41463)
0929 10:33:38.217998 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:38.218004 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:38.218688 (+ 684us) server_negotiation.cc:407] Connection header received
0929 10:33:38.218788 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:38.218790 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:38.218828 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:38.218893 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:38.219408 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:38.219912 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:38.221156 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:38.221359 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:38.221437 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:38.222883 (+ 1446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:38.222887 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:38.222889 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:38.222911 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:38.224792 (+ 1881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:38.225302 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:38.225309 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:38.225312 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:38.225410 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:38.225729 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:38.225732 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:38.225734 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:38.226062 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:38.226184 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:38.226486 (+ 302us) server_negotiation.cc:299] Negotiation successful
0929 10:33:38.226582 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.82.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.82.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/krb5kdc/oryx_127.15.82.67.keytab.
Entry for principal oryx/127.15.82.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/krb5kdc/oryx_127.15.82.67.keytab.
I20260929 10:33:38.245141 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.67:0
--local_ip_for_outbound_sockets=127.15.82.67
--webserver_interface=127.15.82.67
--webserver_port=0
--tserver_master_addrs=127.15.82.126:41463
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/krb5kdc/oryx_127.15.82.67.keytab
--principal=oryx/127.15.82.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:43047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:33:38.359179 23563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:33:38.359498 23563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:33:38.359586 23563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:33:38.363739 23563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:33:38.363827 23563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:33:38.363896 23563 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 10:33:38.363981 23563 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.67
I20260929 10:33:38.369506 23563 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:43047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/krb5kdc/oryx_127.15.82.67.keytab
--principal=oryx/127.15.82.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.82.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.82.126:41463
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23563
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.67
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:33:38.370790 23563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:33:38.371891 23563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:33:38.382719 23571 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:38.382699 23569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:38.382730 23568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:38.382943 23563 server_base.cc:1034] running on GCE node
I20260929 10:33:38.383503 23563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:33:38.384164 23563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:33:38.385325 23563 hybrid_clock.cc:648] HybridClock initialized: now 1790678018385308 us; error 29 us; skew 500 ppm
Sep 29 10:33:38 dist-test-slave-vg89 krb5kdc[23166](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678018, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.82.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:33:38.388155 23563 init.cc:377] Logged in from keytab as oryx/127.15.82.67@KRBTEST.COM (short username oryx)
I20260929 10:33:38.389029 23563 webserver.cc:460] Webserver started at http://127.15.82.67:33331/ using document root <none> and password file <none>
I20260929 10:33:38.389616 23563 fs_manager.cc:359] Metadata directory not provided
I20260929 10:33:38.389708 23563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:33:38.389942 23563 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:33:38.392144 23563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/ts-2/data/instance:
uuid: "bf412a3ffd4e41f5b81db3be3dfa6462"
format_stamp: "Formatted at 2026-09-29 10:33:38 on dist-test-slave-vg89"
server_key: "4717e74597dec347a9d7f8a0b02778b1"
server_key_iv: "ea23c5bff293a31098a350b4b306c5ed"
server_key_version: "encryptionkey@0"
I20260929 10:33:38.392716 23563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/ts-2/wal/instance:
uuid: "bf412a3ffd4e41f5b81db3be3dfa6462"
format_stamp: "Formatted at 2026-09-29 10:33:38 on dist-test-slave-vg89"
server_key: "4717e74597dec347a9d7f8a0b02778b1"
server_key_iv: "ea23c5bff293a31098a350b4b306c5ed"
server_key_version: "encryptionkey@0"
I20260929 10:33:38.396601 23563 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260929 10:33:38.399152 23578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:38.400415 23563 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260929 10:33:38.400557 23563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/ts-2/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/ts-2/wal
uuid: "bf412a3ffd4e41f5b81db3be3dfa6462"
format_stamp: "Formatted at 2026-09-29 10:33:38 on dist-test-slave-vg89"
server_key: "4717e74597dec347a9d7f8a0b02778b1"
server_key_iv: "ea23c5bff293a31098a350b4b306c5ed"
server_key_version: "encryptionkey@0"
I20260929 10:33:38.400689 23563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:33:38.433317 23563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:33:38.436407 23563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:33:38.436626 23563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:33:38.438105 23563 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:33:38.438181 23563 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:38.438256 23563 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:33:38.438300 23563 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 10:33:38 dist-test-slave-vg89 krb5kdc[23166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678018, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.82.67@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM
I20260929 10:33:38.449303 23563 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.67:36501
I20260929 10:33:38.449409 23693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.67:36501 every 8 connection(s)
I20260929 10:33:38.450449 23563 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
I20260929 10:33:38.452226 23260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:38.439068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.67:41419 (local address 127.15.82.126:41463)
0929 10:33:38.439203 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:38.439207 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:38.440283 (+ 1076us) server_negotiation.cc:407] Connection header received
0929 10:33:38.441531 (+ 1248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:38.441537 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:38.441610 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:38.441721 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:38.443066 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:38.443619 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:38.445000 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:38.445424 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:38.445556 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:38.448290 (+ 2734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:38.448294 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:38.448297 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:38.448331 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:38.449963 (+ 1632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:38.450472 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:38.450477 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:38.450478 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:38.450574 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:38.450925 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:38.450928 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:38.450930 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:38.451165 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:38.451266 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:38.451859 (+ 593us) server_negotiation.cc:299] Negotiation successful
0929 10:33:38.452011 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260929 10:33:38.452234 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 23563
I20260929 10:33:38.452358 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/ts-2/wal/instance
I20260929 10:33:38.452661 15689 external_mini_cluster.cc:1428] Setting key 6d3dcd6fbdf4e96d83fdd28a9a0d529b
I20260929 10:33:38.452953 23587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:38.439627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:41463 (local address 127.15.82.67:41419)
0929 10:33:38.440126 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:38.440170 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:38.441227 (+ 1057us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:38.441938 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:38.441950 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:38.442419 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:38.442907 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:38.442921 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:38.443764 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:38.443768 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:38.444836 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:38.444848 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:38.445615 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:38.445621 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:38.445858 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:38.446531 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:38.446550 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:38.448120 (+ 1570us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:38.450078 (+ 1958us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:38.450082 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:38.450097 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:38.450337 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:38.450675 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:38.450677 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:38.450678 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:38.450807 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:38.451317 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:38.451323 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:38.451607 (+ 284us) client_negotiation.cc:769] Sending connection context
0929 10:33:38.451794 (+ 187us) client_negotiation.cc:240] Negotiation successful
0929 10:33:38.452035 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":123,"threads_started":1}
I20260929 10:33:38.454048 23694 heartbeater.cc:347] Connected to a master server at 127.15.82.126:41463
I20260929 10:33:38.454193 23694 heartbeater.cc:464] Registering TS with master...
I20260929 10:33:38.454797 23694 heartbeater.cc:511] Master 127.15.82.126:41463 requested a full tablet report, sending...
I20260929 10:33:38.455787 23210 ts_manager.cc:194] Registered new tserver with Master: bf412a3ffd4e41f5b81db3be3dfa6462 (127.15.82.67:36501)
I20260929 10:33:38.456513 23210 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.82.67@KRBTEST.COM'} at 127.15.82.67:41419
I20260929 10:33:38.456835 15689 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 10:33:38.465521 23700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:38.456178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:41463 (local address 127.15.82.67:36215)
0929 10:33:38.456450 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:38.456464 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:38.456545 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:38.456942 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:38.456945 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:38.457169 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:38.457265 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:38.457270 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:38.458286 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:38.458292 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:38.459233 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:38.459242 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:38.459696 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:38.459699 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:38.459789 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:38.460326 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:38.460337 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:38.460634 (+ 297us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:38.463941 (+ 3307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:38.463944 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:38.463946 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:38.464148 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:38.464469 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:38.464472 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:38.464476 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:38.464558 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:38.464949 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:38.464952 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:38.465052 (+ 100us) client_negotiation.cc:769] Sending connection context
0929 10:33:38.465169 (+ 117us) client_negotiation.cc:240] Negotiation successful
0929 10:33:38.465322 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":101,"threads_started":1}
I20260929 10:33:38.465514 23260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:38.456641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.67:36215 (local address 127.15.82.126:41463)
0929 10:33:38.456739 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:38.456744 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:38.456755 (+ 11us) server_negotiation.cc:407] Connection header received
0929 10:33:38.456789 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:38.456792 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:38.456830 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:38.456912 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:38.457383 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:38.458134 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:38.459354 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:38.459574 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:38.459698 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:38.460771 (+ 1073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:38.460777 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:38.460783 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:38.460813 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:38.462489 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:38.464266 (+ 1777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:38.464271 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:38.464273 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:38.464363 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:38.464654 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:38.464658 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:38.464659 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:38.464849 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:38.464961 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:38.465227 (+ 266us) server_negotiation.cc:299] Negotiation successful
0929 10:33:38.465344 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
Sep 29 10:33:38 dist-test-slave-vg89 krb5kdc[23166](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678018, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 29 10:33:38 dist-test-slave-vg89 krb5kdc[23166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:33:38 dist-test-slave-vg89 krb5kdc[23166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
W20260929 10:33:38.482529 23260 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:33:38.476358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35996 (local address 127.15.82.126:41463)
0929 10:33:38.476507 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:38.476511 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:38.476525 (+ 14us) server_negotiation.cc:407] Connection header received
0929 10:33:38.476588 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:38.476591 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:38.476642 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:38.476713 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:38.477506 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:38.478036 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:38.478920 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:38.479162 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:38.479248 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:38.482291 (+ 3043us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35996: BlockingRecv error: recv got EOF from 127.0.0.1:35996 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Sep 29 10:33:38 dist-test-slave-vg89 krb5kdc[23166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678018, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM
I20260929 10:33:38.491468 23260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:38.483900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36008 (local address 127.15.82.126:41463)
0929 10:33:38.484041 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:38.484045 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:38.484184 (+ 139us) server_negotiation.cc:407] Connection header received
0929 10:33:38.484317 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:38.484320 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:38.484368 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:38.484450 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:38.485023 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:38.485522 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:38.486321 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:38.486620 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:38.486726 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:38.488798 (+ 2072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:38.488801 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:38.488802 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:38.488826 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:38.490027 (+ 1201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:38.490492 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:38.490494 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:38.490496 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:38.490578 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:38.490811 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:38.490813 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:38.490814 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:38.490976 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:38.491079 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:38.491264 (+ 185us) server_negotiation.cc:299] Negotiation successful
0929 10:33:38.491351 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260929 10:33:38.494248 23210 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36008:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 10:33:38.497475 23210 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:33:38.512619 23719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:38.507201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.66:34377 (local address 127.0.0.1:50570)
0929 10:33:38.507438 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:38.507452 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:38.507532 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:38.508600 (+ 1068us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:38.508604 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:38.508621 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:33:38.508699 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:38.508704 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:38.510077 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:38.510080 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:38.511039 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:38.511049 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:38.511975 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:38.511979 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:38.512085 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:38.512188 (+ 103us) client_negotiation.cc:769] Sending connection context
0929 10:33:38.512323 (+ 135us) client_negotiation.cc:240] Negotiation successful
0929 10:33:38.512462 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"mutex_wait_us":27,"thread_start_us":86,"threads_started":1}
I20260929 10:33:38.512632 23721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:38.507636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50570 (local address 127.15.82.66:34377)
0929 10:33:38.508157 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:38.508174 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:38.508198 (+ 24us) server_negotiation.cc:407] Connection header received
0929 10:33:38.508272 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:38.508276 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:38.508446 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:38.508845 (+ 399us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:33:38.508972 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:38.509966 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:38.511174 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:38.511846 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:38.511977 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:38.512169 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:38.512369 (+ 200us) server_negotiation.cc:299] Negotiation successful
0929 10:33:38.512466 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":106,"threads_started":1}
I20260929 10:33:38.513202 23717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:38.506414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.67:36501 (local address 127.0.0.1:60164)
0929 10:33:38.506842 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:38.506900 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:38.507040 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:38.508157 (+ 1117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:38.508160 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:38.508179 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:33:38.508302 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:38.508308 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:38.509458 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:38.509462 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:38.510996 (+ 1534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:38.511003 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:38.512102 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:38.512106 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:38.512883 (+ 777us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:38.512901 (+ 18us) client_negotiation.cc:769] Sending connection context
0929 10:33:38.512993 (+ 92us) client_negotiation.cc:240] Negotiation successful
0929 10:33:38.513069 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"mutex_wait_us":699,"thread_start_us":94,"threads_started":1}
I20260929 10:33:38.513263 23720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:38.507234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60164 (local address 127.15.82.67:36501)
0929 10:33:38.507681 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:38.507694 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:38.507717 (+ 23us) server_negotiation.cc:407] Connection header received
0929 10:33:38.507801 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:38.507806 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:38.507997 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:38.508453 (+ 456us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:33:38.508605 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:38.509326 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:38.511144 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:38.511955 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:38.512157 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:38.512324 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:38.513029 (+ 705us) server_negotiation.cc:299] Negotiation successful
0929 10:33:38.513130 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":121,"threads_started":1}
I20260929 10:33:38.514258 23716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:38.506451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.65:34763 (local address 127.0.0.1:40002)
0929 10:33:38.506980 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:38.506992 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:38.507055 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:38.507945 (+ 890us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:38.507952 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:38.507980 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:33:38.508133 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:38.508141 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:38.509743 (+ 1602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:38.509747 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:38.510992 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:38.511003 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:38.512102 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:38.512106 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:38.513955 (+ 1849us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:38.513977 (+ 22us) client_negotiation.cc:769] Sending connection context
0929 10:33:38.514047 (+ 70us) client_negotiation.cc:240] Negotiation successful
0929 10:33:38.514121 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"mutex_wait_us":1758,"thread_start_us":90,"threads_started":1}
I20260929 10:33:38.514814 23718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:38.506600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40002 (local address 127.15.82.65:34763)
0929 10:33:38.507352 (+ 752us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:38.507359 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:38.507387 (+ 28us) server_negotiation.cc:407] Connection header received
0929 10:33:38.507475 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:38.507483 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:38.507757 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:38.508310 (+ 553us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:33:38.508451 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:38.509625 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:38.511152 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:38.511959 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:38.512183 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:38.512306 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:38.514540 (+ 2234us) server_negotiation.cc:299] Negotiation successful
0929 10:33:38.514672 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":623,"thread_start_us":98,"threads_started":1}
I20260929 10:33:38.516340 23628 tablet_service.cc:1467] Processing CreateTablet for tablet e38026ad09c84eb697f769d38e24a893 (DEFAULT_TABLE table=test-table [id=1b4d69e8d5d5444cb8a291665046113a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:33:38.516240 23488 tablet_service.cc:1467] Processing CreateTablet for tablet e38026ad09c84eb697f769d38e24a893 (DEFAULT_TABLE table=test-table [id=1b4d69e8d5d5444cb8a291665046113a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:33:38.517427 23488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e38026ad09c84eb697f769d38e24a893. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:33:38.517427 23628 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e38026ad09c84eb697f769d38e24a893. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:33:38.517511 23348 tablet_service.cc:1467] Processing CreateTablet for tablet e38026ad09c84eb697f769d38e24a893 (DEFAULT_TABLE table=test-table [id=1b4d69e8d5d5444cb8a291665046113a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:33:38.518589 23348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e38026ad09c84eb697f769d38e24a893. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:33:38.525182 23722 tablet_bootstrap.cc:492] T e38026ad09c84eb697f769d38e24a893 P e44ce81184b94969854575ed79d06c48: Bootstrap starting.
I20260929 10:33:38.526721 23724 tablet_bootstrap.cc:492] T e38026ad09c84eb697f769d38e24a893 P c1f0b22653cc422d9b26cb3f6b39b1d7: Bootstrap starting.
I20260929 10:33:38.526726 23723 tablet_bootstrap.cc:492] T e38026ad09c84eb697f769d38e24a893 P bf412a3ffd4e41f5b81db3be3dfa6462: Bootstrap starting.
I20260929 10:33:38.527868 23722 tablet_bootstrap.cc:654] T e38026ad09c84eb697f769d38e24a893 P e44ce81184b94969854575ed79d06c48: Neither blocks nor log segments found. Creating new log.
I20260929 10:33:38.528697 23724 tablet_bootstrap.cc:654] T e38026ad09c84eb697f769d38e24a893 P c1f0b22653cc422d9b26cb3f6b39b1d7: Neither blocks nor log segments found. Creating new log.
I20260929 10:33:38.528771 23723 tablet_bootstrap.cc:654] T e38026ad09c84eb697f769d38e24a893 P bf412a3ffd4e41f5b81db3be3dfa6462: Neither blocks nor log segments found. Creating new log.
I20260929 10:33:38.528765 23722 log.cc:824] T e38026ad09c84eb697f769d38e24a893 P e44ce81184b94969854575ed79d06c48: Log is configured to *not* fsync() on all Append() calls
I20260929 10:33:38.529508 23723 log.cc:824] T e38026ad09c84eb697f769d38e24a893 P bf412a3ffd4e41f5b81db3be3dfa6462: Log is configured to *not* fsync() on all Append() calls
I20260929 10:33:38.529534 23724 log.cc:824] T e38026ad09c84eb697f769d38e24a893 P c1f0b22653cc422d9b26cb3f6b39b1d7: Log is configured to *not* fsync() on all Append() calls
I20260929 10:33:38.530912 23722 tablet_bootstrap.cc:492] T e38026ad09c84eb697f769d38e24a893 P e44ce81184b94969854575ed79d06c48: No bootstrap required, opened a new log
I20260929 10:33:38.531126 23722 ts_tablet_manager.cc:1397] T e38026ad09c84eb697f769d38e24a893 P e44ce81184b94969854575ed79d06c48: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260929 10:33:38.531525 23723 tablet_bootstrap.cc:492] T e38026ad09c84eb697f769d38e24a893 P bf412a3ffd4e41f5b81db3be3dfa6462: No bootstrap required, opened a new log
I20260929 10:33:38.531709 23723 ts_tablet_manager.cc:1397] T e38026ad09c84eb697f769d38e24a893 P bf412a3ffd4e41f5b81db3be3dfa6462: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260929 10:33:38.535709 23724 tablet_bootstrap.cc:492] T e38026ad09c84eb697f769d38e24a893 P c1f0b22653cc422d9b26cb3f6b39b1d7: No bootstrap required, opened a new log
I20260929 10:33:38.535655 23722 raft_consensus.cc:348] T e38026ad09c84eb697f769d38e24a893 P e44ce81184b94969854575ed79d06c48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44ce81184b94969854575ed79d06c48" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34763 } } peers { permanent_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36501 } } peers { permanent_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34377 } }
I20260929 10:33:38.535655 23723 raft_consensus.cc:348] T e38026ad09c84eb697f769d38e24a893 P bf412a3ffd4e41f5b81db3be3dfa6462 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44ce81184b94969854575ed79d06c48" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34763 } } peers { permanent_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36501 } } peers { permanent_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34377 } }
I20260929 10:33:38.535929 23722 raft_consensus.cc:374] T e38026ad09c84eb697f769d38e24a893 P e44ce81184b94969854575ed79d06c48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:33:38.535929 23723 raft_consensus.cc:374] T e38026ad09c84eb697f769d38e24a893 P bf412a3ffd4e41f5b81db3be3dfa6462 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:33:38.535935 23724 ts_tablet_manager.cc:1397] T e38026ad09c84eb697f769d38e24a893 P c1f0b22653cc422d9b26cb3f6b39b1d7: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.000s
I20260929 10:33:38.535990 23723 raft_consensus.cc:729] T e38026ad09c84eb697f769d38e24a893 P bf412a3ffd4e41f5b81db3be3dfa6462 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf412a3ffd4e41f5b81db3be3dfa6462, State: Initialized, Role: FOLLOWER
I20260929 10:33:38.535990 23722 raft_consensus.cc:729] T e38026ad09c84eb697f769d38e24a893 P e44ce81184b94969854575ed79d06c48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e44ce81184b94969854575ed79d06c48, State: Initialized, Role: FOLLOWER
I20260929 10:33:38.536463 23723 consensus_queue.cc:260] T e38026ad09c84eb697f769d38e24a893 P bf412a3ffd4e41f5b81db3be3dfa6462 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44ce81184b94969854575ed79d06c48" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34763 } } peers { permanent_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36501 } } peers { permanent_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34377 } }
I20260929 10:33:38.536463 23722 consensus_queue.cc:260] T e38026ad09c84eb697f769d38e24a893 P e44ce81184b94969854575ed79d06c48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44ce81184b94969854575ed79d06c48" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34763 } } peers { permanent_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36501 } } peers { permanent_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34377 } }
I20260929 10:33:38.537120 23414 heartbeater.cc:503] Master 127.15.82.126:41463 was elected leader, sending a full tablet report...
I20260929 10:33:38.537487 23722 ts_tablet_manager.cc:1428] T e38026ad09c84eb697f769d38e24a893 P e44ce81184b94969854575ed79d06c48: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260929 10:33:38.537487 23723 ts_tablet_manager.cc:1428] T e38026ad09c84eb697f769d38e24a893 P bf412a3ffd4e41f5b81db3be3dfa6462: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260929 10:33:38.537853 23694 heartbeater.cc:503] Master 127.15.82.126:41463 was elected leader, sending a full tablet report...
I20260929 10:33:38.539788 23724 raft_consensus.cc:348] T e38026ad09c84eb697f769d38e24a893 P c1f0b22653cc422d9b26cb3f6b39b1d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44ce81184b94969854575ed79d06c48" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34763 } } peers { permanent_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36501 } } peers { permanent_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34377 } }
I20260929 10:33:38.540074 23724 raft_consensus.cc:374] T e38026ad09c84eb697f769d38e24a893 P c1f0b22653cc422d9b26cb3f6b39b1d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:33:38.540151 23724 raft_consensus.cc:729] T e38026ad09c84eb697f769d38e24a893 P c1f0b22653cc422d9b26cb3f6b39b1d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1f0b22653cc422d9b26cb3f6b39b1d7, State: Initialized, Role: FOLLOWER
I20260929 10:33:38.540654 23724 consensus_queue.cc:260] T e38026ad09c84eb697f769d38e24a893 P c1f0b22653cc422d9b26cb3f6b39b1d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44ce81184b94969854575ed79d06c48" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34763 } } peers { permanent_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36501 } } peers { permanent_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34377 } }
I20260929 10:33:38.541723 23724 ts_tablet_manager.cc:1428] T e38026ad09c84eb697f769d38e24a893 P c1f0b22653cc422d9b26cb3f6b39b1d7: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260929 10:33:38.541842 23554 heartbeater.cc:503] Master 127.15.82.126:41463 was elected leader, sending a full tablet report...
W20260929 10:33:38.701519 23695 tablet.cc:2369] T e38026ad09c84eb697f769d38e24a893 P bf412a3ffd4e41f5b81db3be3dfa6462: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 10:33:38.711905 23555 tablet.cc:2369] T e38026ad09c84eb697f769d38e24a893 P c1f0b22653cc422d9b26cb3f6b39b1d7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 10:33:38.725489 23415 tablet.cc:2369] T e38026ad09c84eb697f769d38e24a893 P e44ce81184b94969854575ed79d06c48: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 10:33:38.824383 23729 raft_consensus.cc:482] T e38026ad09c84eb697f769d38e24a893 P bf412a3ffd4e41f5b81db3be3dfa6462 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:33:38.824651 23729 raft_consensus.cc:504] T e38026ad09c84eb697f769d38e24a893 P bf412a3ffd4e41f5b81db3be3dfa6462 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44ce81184b94969854575ed79d06c48" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34763 } } peers { permanent_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36501 } } peers { permanent_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34377 } }
I20260929 10:33:38.825856 23729 leader_election.cc:288] T e38026ad09c84eb697f769d38e24a893 P bf412a3ffd4e41f5b81db3be3dfa6462 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e44ce81184b94969854575ed79d06c48 (127.15.82.65:34763), c1f0b22653cc422d9b26cb3f6b39b1d7 (127.15.82.66:34377)
I20260929 10:33:38.830598 23587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:38.826206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.65:34763 (local address 127.15.82.67:45069)
0929 10:33:38.826369 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:38.826388 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:38.826522 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:38.826860 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:38.826864 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:38.826884 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:33:38.827024 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:38.827031 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:38.828117 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:38.828121 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:38.829156 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:38.829165 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:38.830099 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:38.830102 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:38.830210 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:38.830236 (+ 26us) client_negotiation.cc:769] Sending connection context
0929 10:33:38.830315 (+ 79us) client_negotiation.cc:240] Negotiation successful
0929 10:33:38.830389 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260929 10:33:38.830614 23718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:38.826313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.67:45069 (local address 127.15.82.65:34763)
0929 10:33:38.826490 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:38.826494 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:38.826510 (+ 16us) server_negotiation.cc:407] Connection header received
0929 10:33:38.826662 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:38.826665 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:38.826727 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:38.826806 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:33:38.827172 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:38.828004 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:38.829286 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:38.829962 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:38.830055 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:38.830100 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:38.830341 (+ 241us) server_negotiation.cc:299] Negotiation successful
0929 10:33:38.830447 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260929 10:33:38.830976 23731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:38.826566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.66:34377 (local address 127.15.82.67:59793)
0929 10:33:38.826926 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:38.826939 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:38.827028 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:38.827261 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:38.827264 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:38.827280 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:33:38.827813 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:38.827822 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:38.828774 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:38.828776 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:38.829745 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:38.829755 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:38.830563 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:38.830567 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:38.830668 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:38.830698 (+ 30us) client_negotiation.cc:769] Sending connection context
0929 10:33:38.830778 (+ 80us) client_negotiation.cc:240] Negotiation successful
0929 10:33:38.830846 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":117,"threads_started":1}
I20260929 10:33:38.830986 23721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:38.826696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.67:59793 (local address 127.15.82.66:34377)
0929 10:33:38.826830 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:38.826836 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:38.827033 (+ 197us) server_negotiation.cc:407] Connection header received
0929 10:33:38.827095 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:38.827100 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:38.827157 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:38.827221 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:33:38.827928 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:38.828658 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:38.829870 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:38.830448 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:38.830552 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:38.830612 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:38.830803 (+ 191us) server_negotiation.cc:299] Negotiation successful
0929 10:33:38.830878 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 10:33:38.831393 23368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e38026ad09c84eb697f769d38e24a893" candidate_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e44ce81184b94969854575ed79d06c48" is_pre_election: true
I20260929 10:33:38.831763 23368 raft_consensus.cc:2454] T e38026ad09c84eb697f769d38e24a893 P e44ce81184b94969854575ed79d06c48 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf412a3ffd4e41f5b81db3be3dfa6462 in term 0.
I20260929 10:33:38.832170 23505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e38026ad09c84eb697f769d38e24a893" candidate_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" is_pre_election: true
I20260929 10:33:38.832304 23581 leader_election.cc:302] T e38026ad09c84eb697f769d38e24a893 P bf412a3ffd4e41f5b81db3be3dfa6462 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bf412a3ffd4e41f5b81db3be3dfa6462, e44ce81184b94969854575ed79d06c48; no voters:
I20260929 10:33:38.832463 23505 raft_consensus.cc:2454] T e38026ad09c84eb697f769d38e24a893 P c1f0b22653cc422d9b26cb3f6b39b1d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf412a3ffd4e41f5b81db3be3dfa6462 in term 0.
I20260929 10:33:38.832626 23729 raft_consensus.cc:2781] T e38026ad09c84eb697f769d38e24a893 P bf412a3ffd4e41f5b81db3be3dfa6462 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:33:38.832737 23729 raft_consensus.cc:482] T e38026ad09c84eb697f769d38e24a893 P bf412a3ffd4e41f5b81db3be3dfa6462 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:33:38.832801 23729 raft_consensus.cc:3037] T e38026ad09c84eb697f769d38e24a893 P bf412a3ffd4e41f5b81db3be3dfa6462 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:33:38.833904 23729 raft_consensus.cc:504] T e38026ad09c84eb697f769d38e24a893 P bf412a3ffd4e41f5b81db3be3dfa6462 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44ce81184b94969854575ed79d06c48" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34763 } } peers { permanent_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36501 } } peers { permanent_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34377 } }
I20260929 10:33:38.834245 23729 leader_election.cc:288] T e38026ad09c84eb697f769d38e24a893 P bf412a3ffd4e41f5b81db3be3dfa6462 [CANDIDATE]: Term 1 election: Requested vote from peers e44ce81184b94969854575ed79d06c48 (127.15.82.65:34763), c1f0b22653cc422d9b26cb3f6b39b1d7 (127.15.82.66:34377)
I20260929 10:33:38.834640 23505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e38026ad09c84eb697f769d38e24a893" candidate_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7"
I20260929 10:33:38.834651 23368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e38026ad09c84eb697f769d38e24a893" candidate_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e44ce81184b94969854575ed79d06c48"
I20260929 10:33:38.834767 23368 raft_consensus.cc:3037] T e38026ad09c84eb697f769d38e24a893 P e44ce81184b94969854575ed79d06c48 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:33:38.834767 23505 raft_consensus.cc:3037] T e38026ad09c84eb697f769d38e24a893 P c1f0b22653cc422d9b26cb3f6b39b1d7 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:33:38.836037 23505 raft_consensus.cc:2454] T e38026ad09c84eb697f769d38e24a893 P c1f0b22653cc422d9b26cb3f6b39b1d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf412a3ffd4e41f5b81db3be3dfa6462 in term 1.
I20260929 10:33:38.836037 23368 raft_consensus.cc:2454] T e38026ad09c84eb697f769d38e24a893 P e44ce81184b94969854575ed79d06c48 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf412a3ffd4e41f5b81db3be3dfa6462 in term 1.
I20260929 10:33:38.836338 23581 leader_election.cc:302] T e38026ad09c84eb697f769d38e24a893 P bf412a3ffd4e41f5b81db3be3dfa6462 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bf412a3ffd4e41f5b81db3be3dfa6462, e44ce81184b94969854575ed79d06c48; no voters:
I20260929 10:33:38.836493 23729 raft_consensus.cc:2781] T e38026ad09c84eb697f769d38e24a893 P bf412a3ffd4e41f5b81db3be3dfa6462 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:33:38.836709 23729 raft_consensus.cc:686] T e38026ad09c84eb697f769d38e24a893 P bf412a3ffd4e41f5b81db3be3dfa6462 [term 1 LEADER]: Becoming Leader. State: Replica: bf412a3ffd4e41f5b81db3be3dfa6462, State: Running, Role: LEADER
I20260929 10:33:38.837041 23729 consensus_queue.cc:237] T e38026ad09c84eb697f769d38e24a893 P bf412a3ffd4e41f5b81db3be3dfa6462 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44ce81184b94969854575ed79d06c48" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34763 } } peers { permanent_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36501 } } peers { permanent_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34377 } }
I20260929 10:33:38.840399 23209 catalog_manager.cc:5495] T e38026ad09c84eb697f769d38e24a893 P bf412a3ffd4e41f5b81db3be3dfa6462 reported cstate change: term changed from 0 to 1, leader changed from <none> to bf412a3ffd4e41f5b81db3be3dfa6462 (127.15.82.67). New cstate: current_term: 1 leader_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44ce81184b94969854575ed79d06c48" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34763 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36501 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34377 } health_report { overall_health: UNKNOWN } } }
I20260929 10:33:38.880224 23720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:38.876719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60172 (local address 127.15.82.67:36501)
0929 10:33:38.876852 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:38.876855 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:38.876869 (+ 14us) server_negotiation.cc:407] Connection header received
0929 10:33:38.876968 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:38.876970 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:38.877024 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:38.877114 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:33:38.877373 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:38.877946 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:38.878831 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:38.879059 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:38.879138 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:38.879371 (+ 233us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:33:38.879830 (+ 459us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:33:38.879916 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:38.879979 (+ 63us) server_negotiation.cc:299] Negotiation successful
0929 10:33:38.880028 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 10:33:38.886468 23368 raft_consensus.cc:1261] T e38026ad09c84eb697f769d38e24a893 P e44ce81184b94969854575ed79d06c48 [term 1 FOLLOWER]: Refusing update from remote peer bf412a3ffd4e41f5b81db3be3dfa6462: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:33:38.886499 23505 raft_consensus.cc:1261] T e38026ad09c84eb697f769d38e24a893 P c1f0b22653cc422d9b26cb3f6b39b1d7 [term 1 FOLLOWER]: Refusing update from remote peer bf412a3ffd4e41f5b81db3be3dfa6462: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:33:38.887249 23732 consensus_queue.cc:1035] T e38026ad09c84eb697f769d38e24a893 P bf412a3ffd4e41f5b81db3be3dfa6462 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:33:38.887454 23729 consensus_queue.cc:1035] T e38026ad09c84eb697f769d38e24a893 P bf412a3ffd4e41f5b81db3be3dfa6462 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e44ce81184b94969854575ed79d06c48" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:33:38.896021 23736 mvcc.cc:205] Tried to move back new op lower bound from 7334617165351206912 to 7334617165159202816. Current Snapshot: MvccSnapshot[applied={T|T < 7334617165351206912}]
I20260929 10:33:38.904210 23210 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36008:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:33:38.904399 23210 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36008:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:33:38.907207 23210 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f5a8c72fbf5a49f0a0dbf4031fe3ff93: Sending DeleteTablet for 3 replicas of tablet e38026ad09c84eb697f769d38e24a893
I20260929 10:33:38.908079 23348 tablet_service.cc:1514] Processing DeleteTablet for tablet e38026ad09c84eb697f769d38e24a893 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:33:38 UTC) from {username='oryx', principal='oryx/127.15.82.126@KRBTEST.COM'} at 127.0.0.1:40002
I20260929 10:33:38.908129 23488 tablet_service.cc:1514] Processing DeleteTablet for tablet e38026ad09c84eb697f769d38e24a893 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:33:38 UTC) from {username='oryx', principal='oryx/127.15.82.126@KRBTEST.COM'} at 127.0.0.1:50570
I20260929 10:33:38.908196 23628 tablet_service.cc:1514] Processing DeleteTablet for tablet e38026ad09c84eb697f769d38e24a893 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:33:38 UTC) from {username='oryx', principal='oryx/127.15.82.126@KRBTEST.COM'} at 127.0.0.1:60164
I20260929 10:33:38.908625 23746 tablet_replica.cc:330] T e38026ad09c84eb697f769d38e24a893 P e44ce81184b94969854575ed79d06c48: stopping tablet replica
I20260929 10:33:38.908803 23745 tablet_replica.cc:330] T e38026ad09c84eb697f769d38e24a893 P c1f0b22653cc422d9b26cb3f6b39b1d7: stopping tablet replica
I20260929 10:33:38.908985 23746 raft_consensus.cc:2229] T e38026ad09c84eb697f769d38e24a893 P e44ce81184b94969854575ed79d06c48 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:33:38.909181 23745 raft_consensus.cc:2229] T e38026ad09c84eb697f769d38e24a893 P c1f0b22653cc422d9b26cb3f6b39b1d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:33:38.909336 23746 raft_consensus.cc:2258] T e38026ad09c84eb697f769d38e24a893 P e44ce81184b94969854575ed79d06c48 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:33:38.909546 23745 raft_consensus.cc:2258] T e38026ad09c84eb697f769d38e24a893 P c1f0b22653cc422d9b26cb3f6b39b1d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:33:38.909533 23210 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36008:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 10:33:38.909739 23747 tablet_replica.cc:330] T e38026ad09c84eb697f769d38e24a893 P bf412a3ffd4e41f5b81db3be3dfa6462: stopping tablet replica
W20260929 10:33:38.910004 23210 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:33:38.910027 23747 raft_consensus.cc:2229] T e38026ad09c84eb697f769d38e24a893 P bf412a3ffd4e41f5b81db3be3dfa6462 [term 1 LEADER]: Raft consensus shutting down.
I20260929 10:33:38.910463 23747 raft_consensus.cc:2258] T e38026ad09c84eb697f769d38e24a893 P bf412a3ffd4e41f5b81db3be3dfa6462 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:33:38.911224 23746 ts_tablet_manager.cc:1905] T e38026ad09c84eb697f769d38e24a893 P e44ce81184b94969854575ed79d06c48: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:33:38.911934 23747 ts_tablet_manager.cc:1905] T e38026ad09c84eb697f769d38e24a893 P bf412a3ffd4e41f5b81db3be3dfa6462: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:33:38.914204 23746 ts_tablet_manager.cc:1918] T e38026ad09c84eb697f769d38e24a893 P e44ce81184b94969854575ed79d06c48: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:33:38.914314 23746 log.cc:1196] T e38026ad09c84eb697f769d38e24a893 P e44ce81184b94969854575ed79d06c48: Deleting WAL directory at /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/ts-0/wal/wals/e38026ad09c84eb697f769d38e24a893
I20260929 10:33:38.914637 23747 ts_tablet_manager.cc:1918] T e38026ad09c84eb697f769d38e24a893 P bf412a3ffd4e41f5b81db3be3dfa6462: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:33:38.914705 23746 ts_tablet_manager.cc:1939] T e38026ad09c84eb697f769d38e24a893 P e44ce81184b94969854575ed79d06c48: Deleting consensus metadata
I20260929 10:33:38.914739 23747 log.cc:1196] T e38026ad09c84eb697f769d38e24a893 P bf412a3ffd4e41f5b81db3be3dfa6462: Deleting WAL directory at /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/ts-2/wal/wals/e38026ad09c84eb697f769d38e24a893
I20260929 10:33:38.915086 23747 ts_tablet_manager.cc:1939] T e38026ad09c84eb697f769d38e24a893 P bf412a3ffd4e41f5b81db3be3dfa6462: Deleting consensus metadata
I20260929 10:33:38.915693 23348 tablet_service.cc:1467] Processing CreateTablet for tablet cb730556d9f04d749be50a4aba18ab1e (DEFAULT_TABLE table=test-table [id=7d2df19a126a4bbda5f01dfe4d014430]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:33:38.915793 23488 tablet_service.cc:1467] Processing CreateTablet for tablet cb730556d9f04d749be50a4aba18ab1e (DEFAULT_TABLE table=test-table [id=7d2df19a126a4bbda5f01dfe4d014430]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:33:38.915977 23348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb730556d9f04d749be50a4aba18ab1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:33:38.916101 23488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb730556d9f04d749be50a4aba18ab1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:33:38.917687 23745 ts_tablet_manager.cc:1905] T e38026ad09c84eb697f769d38e24a893 P c1f0b22653cc422d9b26cb3f6b39b1d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:33:38.918349 23722 tablet_bootstrap.cc:492] T cb730556d9f04d749be50a4aba18ab1e P e44ce81184b94969854575ed79d06c48: Bootstrap starting.
I20260929 10:33:38.919199 23628 tablet_service.cc:1467] Processing CreateTablet for tablet cb730556d9f04d749be50a4aba18ab1e (DEFAULT_TABLE table=test-table [id=7d2df19a126a4bbda5f01dfe4d014430]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:33:38.919334 23722 tablet_bootstrap.cc:654] T cb730556d9f04d749be50a4aba18ab1e P e44ce81184b94969854575ed79d06c48: Neither blocks nor log segments found. Creating new log.
I20260929 10:33:38.919492 23628 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb730556d9f04d749be50a4aba18ab1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:33:38.920150 23745 ts_tablet_manager.cc:1918] T e38026ad09c84eb697f769d38e24a893 P c1f0b22653cc422d9b26cb3f6b39b1d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:33:38.920250 23745 log.cc:1196] T e38026ad09c84eb697f769d38e24a893 P c1f0b22653cc422d9b26cb3f6b39b1d7: Deleting WAL directory at /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/ts-1/wal/wals/e38026ad09c84eb697f769d38e24a893
I20260929 10:33:38.920320 23197 catalog_manager.cc:4841] TS e44ce81184b94969854575ed79d06c48 (127.15.82.65:34763): tablet e38026ad09c84eb697f769d38e24a893 (table test-table [id=1b4d69e8d5d5444cb8a291665046113a]) successfully deleted
I20260929 10:33:38.920554 23745 ts_tablet_manager.cc:1939] T e38026ad09c84eb697f769d38e24a893 P c1f0b22653cc422d9b26cb3f6b39b1d7: Deleting consensus metadata
I20260929 10:33:38.921311 23724 tablet_bootstrap.cc:492] T cb730556d9f04d749be50a4aba18ab1e P c1f0b22653cc422d9b26cb3f6b39b1d7: Bootstrap starting.
I20260929 10:33:38.921732 23196 catalog_manager.cc:4841] TS bf412a3ffd4e41f5b81db3be3dfa6462 (127.15.82.67:36501): tablet e38026ad09c84eb697f769d38e24a893 (table test-table [id=1b4d69e8d5d5444cb8a291665046113a]) successfully deleted
I20260929 10:33:38.922287 23724 tablet_bootstrap.cc:654] T cb730556d9f04d749be50a4aba18ab1e P c1f0b22653cc422d9b26cb3f6b39b1d7: Neither blocks nor log segments found. Creating new log.
I20260929 10:33:38.923105 23723 tablet_bootstrap.cc:492] T cb730556d9f04d749be50a4aba18ab1e P bf412a3ffd4e41f5b81db3be3dfa6462: Bootstrap starting.
I20260929 10:33:38.924043 23724 tablet_bootstrap.cc:492] T cb730556d9f04d749be50a4aba18ab1e P c1f0b22653cc422d9b26cb3f6b39b1d7: No bootstrap required, opened a new log
I20260929 10:33:38.924070 23723 tablet_bootstrap.cc:654] T cb730556d9f04d749be50a4aba18ab1e P bf412a3ffd4e41f5b81db3be3dfa6462: Neither blocks nor log segments found. Creating new log.
I20260929 10:33:38.924124 23724 ts_tablet_manager.cc:1397] T cb730556d9f04d749be50a4aba18ab1e P c1f0b22653cc422d9b26cb3f6b39b1d7: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260929 10:33:38.924611 23724 raft_consensus.cc:348] T cb730556d9f04d749be50a4aba18ab1e P c1f0b22653cc422d9b26cb3f6b39b1d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44ce81184b94969854575ed79d06c48" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34763 } } peers { permanent_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34377 } } peers { permanent_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36501 } }
I20260929 10:33:38.924721 23724 raft_consensus.cc:374] T cb730556d9f04d749be50a4aba18ab1e P c1f0b22653cc422d9b26cb3f6b39b1d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:33:38.924749 23724 raft_consensus.cc:729] T cb730556d9f04d749be50a4aba18ab1e P c1f0b22653cc422d9b26cb3f6b39b1d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1f0b22653cc422d9b26cb3f6b39b1d7, State: Initialized, Role: FOLLOWER
I20260929 10:33:38.924834 23197 catalog_manager.cc:4841] TS c1f0b22653cc422d9b26cb3f6b39b1d7 (127.15.82.66:34377): tablet e38026ad09c84eb697f769d38e24a893 (table test-table [id=1b4d69e8d5d5444cb8a291665046113a]) successfully deleted
I20260929 10:33:38.925026 23722 tablet_bootstrap.cc:492] T cb730556d9f04d749be50a4aba18ab1e P e44ce81184b94969854575ed79d06c48: No bootstrap required, opened a new log
I20260929 10:33:38.925132 23722 ts_tablet_manager.cc:1397] T cb730556d9f04d749be50a4aba18ab1e P e44ce81184b94969854575ed79d06c48: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.002s
I20260929 10:33:38.925064 23724 consensus_queue.cc:260] T cb730556d9f04d749be50a4aba18ab1e P c1f0b22653cc422d9b26cb3f6b39b1d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44ce81184b94969854575ed79d06c48" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34763 } } peers { permanent_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34377 } } peers { permanent_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36501 } }
I20260929 10:33:38.925985 23722 raft_consensus.cc:348] T cb730556d9f04d749be50a4aba18ab1e P e44ce81184b94969854575ed79d06c48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44ce81184b94969854575ed79d06c48" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34763 } } peers { permanent_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34377 } } peers { permanent_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36501 } }
I20260929 10:33:38.926116 23722 raft_consensus.cc:374] T cb730556d9f04d749be50a4aba18ab1e P e44ce81184b94969854575ed79d06c48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:33:38.926160 23722 raft_consensus.cc:729] T cb730556d9f04d749be50a4aba18ab1e P e44ce81184b94969854575ed79d06c48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e44ce81184b94969854575ed79d06c48, State: Initialized, Role: FOLLOWER
I20260929 10:33:38.926254 23724 ts_tablet_manager.cc:1428] T cb730556d9f04d749be50a4aba18ab1e P c1f0b22653cc422d9b26cb3f6b39b1d7: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260929 10:33:38.926291 23722 consensus_queue.cc:260] T cb730556d9f04d749be50a4aba18ab1e P e44ce81184b94969854575ed79d06c48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44ce81184b94969854575ed79d06c48" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34763 } } peers { permanent_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34377 } } peers { permanent_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36501 } }
I20260929 10:33:38.926614 23722 ts_tablet_manager.cc:1428] T cb730556d9f04d749be50a4aba18ab1e P e44ce81184b94969854575ed79d06c48: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260929 10:33:38.927789 23723 tablet_bootstrap.cc:492] T cb730556d9f04d749be50a4aba18ab1e P bf412a3ffd4e41f5b81db3be3dfa6462: No bootstrap required, opened a new log
I20260929 10:33:38.927878 23723 ts_tablet_manager.cc:1397] T cb730556d9f04d749be50a4aba18ab1e P bf412a3ffd4e41f5b81db3be3dfa6462: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.001s
I20260929 10:33:38.928385 23723 raft_consensus.cc:348] T cb730556d9f04d749be50a4aba18ab1e P bf412a3ffd4e41f5b81db3be3dfa6462 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44ce81184b94969854575ed79d06c48" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34763 } } peers { permanent_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34377 } } peers { permanent_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36501 } }
I20260929 10:33:38.928499 23723 raft_consensus.cc:374] T cb730556d9f04d749be50a4aba18ab1e P bf412a3ffd4e41f5b81db3be3dfa6462 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:33:38.928534 23723 raft_consensus.cc:729] T cb730556d9f04d749be50a4aba18ab1e P bf412a3ffd4e41f5b81db3be3dfa6462 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf412a3ffd4e41f5b81db3be3dfa6462, State: Initialized, Role: FOLLOWER
I20260929 10:33:38.928656 23723 consensus_queue.cc:260] T cb730556d9f04d749be50a4aba18ab1e P bf412a3ffd4e41f5b81db3be3dfa6462 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44ce81184b94969854575ed79d06c48" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34763 } } peers { permanent_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34377 } } peers { permanent_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36501 } }
I20260929 10:33:38.928956 23723 ts_tablet_manager.cc:1428] T cb730556d9f04d749be50a4aba18ab1e P bf412a3ffd4e41f5b81db3be3dfa6462: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260929 10:33:39.304956 23730 raft_consensus.cc:482] T cb730556d9f04d749be50a4aba18ab1e P c1f0b22653cc422d9b26cb3f6b39b1d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:33:39.305217 23730 raft_consensus.cc:504] T cb730556d9f04d749be50a4aba18ab1e P c1f0b22653cc422d9b26cb3f6b39b1d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44ce81184b94969854575ed79d06c48" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34763 } } peers { permanent_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34377 } } peers { permanent_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36501 } }
I20260929 10:33:39.306354 23730 leader_election.cc:288] T cb730556d9f04d749be50a4aba18ab1e P c1f0b22653cc422d9b26cb3f6b39b1d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e44ce81184b94969854575ed79d06c48 (127.15.82.65:34763), bf412a3ffd4e41f5b81db3be3dfa6462 (127.15.82.67:36501)
I20260929 10:33:39.311981 23718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:39.307031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:56669 (local address 127.15.82.65:34763)
0929 10:33:39.307211 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:39.307218 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:39.307235 (+ 17us) server_negotiation.cc:407] Connection header received
0929 10:33:39.307302 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:39.307305 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:39.307355 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:39.307448 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:33:39.308002 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:39.308782 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:39.310666 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:39.311328 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:39.311436 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:39.311486 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:39.311765 (+ 279us) server_negotiation.cc:299] Negotiation successful
0929 10:33:39.311839 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260929 10:33:39.311981 23720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:39.306795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:47759 (local address 127.15.82.67:36501)
0929 10:33:39.306915 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:39.306920 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:39.307236 (+ 316us) server_negotiation.cc:407] Connection header received
0929 10:33:39.307318 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:39.307321 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:39.307368 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:39.307447 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:33:39.307965 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:39.308782 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:39.310666 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:39.311284 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:39.311425 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:39.311502 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:39.311765 (+ 263us) server_negotiation.cc:299] Negotiation successful
0929 10:33:39.311840 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260929 10:33:39.312192 23751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:39.306706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.65:34763 (local address 127.15.82.66:56669)
0929 10:33:39.306987 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:39.307003 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:39.307132 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:39.307511 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:39.307514 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:39.307561 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:33:39.307790 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:39.307798 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:39.308963 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:39.308966 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:39.310522 (+ 1556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:39.310535 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:39.311445 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:39.311449 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:39.311607 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:39.311655 (+ 48us) client_negotiation.cc:769] Sending connection context
0929 10:33:39.311733 (+ 78us) client_negotiation.cc:240] Negotiation successful
0929 10:33:39.311850 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"mutex_wait_us":303,"spinlock_wait_cycles":10112,"thread_start_us":79,"threads_started":1}
I20260929 10:33:39.312048 23752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:39.306706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.67:36501 (local address 127.15.82.66:47759)
0929 10:33:39.307133 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:39.307145 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:39.307236 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:39.307511 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:39.307514 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:39.307560 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:33:39.307803 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:39.307809 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:39.308946 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:39.308953 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:39.310522 (+ 1569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:39.310535 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:39.311445 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:39.311449 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:39.311612 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:39.311652 (+ 40us) client_negotiation.cc:769] Sending connection context
0929 10:33:39.311733 (+ 81us) client_negotiation.cc:240] Negotiation successful
0929 10:33:39.311852 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"mutex_wait_us":243,"spinlock_wait_cycles":28288,"thread_start_us":115,"threads_started":1}
I20260929 10:33:39.312810 23368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb730556d9f04d749be50a4aba18ab1e" candidate_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e44ce81184b94969854575ed79d06c48" is_pre_election: true
I20260929 10:33:39.312824 23648 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb730556d9f04d749be50a4aba18ab1e" candidate_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" is_pre_election: true
I20260929 10:33:39.313006 23368 raft_consensus.cc:2454] T cb730556d9f04d749be50a4aba18ab1e P e44ce81184b94969854575ed79d06c48 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1f0b22653cc422d9b26cb3f6b39b1d7 in term 0.
I20260929 10:33:39.313129 23648 raft_consensus.cc:2454] T cb730556d9f04d749be50a4aba18ab1e P bf412a3ffd4e41f5b81db3be3dfa6462 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1f0b22653cc422d9b26cb3f6b39b1d7 in term 0.
I20260929 10:33:39.313443 23441 leader_election.cc:302] T cb730556d9f04d749be50a4aba18ab1e P c1f0b22653cc422d9b26cb3f6b39b1d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c1f0b22653cc422d9b26cb3f6b39b1d7, e44ce81184b94969854575ed79d06c48; no voters:
I20260929 10:33:39.313728 23730 raft_consensus.cc:2781] T cb730556d9f04d749be50a4aba18ab1e P c1f0b22653cc422d9b26cb3f6b39b1d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:33:39.313787 23730 raft_consensus.cc:482] T cb730556d9f04d749be50a4aba18ab1e P c1f0b22653cc422d9b26cb3f6b39b1d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:33:39.313825 23730 raft_consensus.cc:3037] T cb730556d9f04d749be50a4aba18ab1e P c1f0b22653cc422d9b26cb3f6b39b1d7 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:33:39.314888 23730 raft_consensus.cc:504] T cb730556d9f04d749be50a4aba18ab1e P c1f0b22653cc422d9b26cb3f6b39b1d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44ce81184b94969854575ed79d06c48" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34763 } } peers { permanent_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34377 } } peers { permanent_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36501 } }
I20260929 10:33:39.315295 23730 leader_election.cc:288] T cb730556d9f04d749be50a4aba18ab1e P c1f0b22653cc422d9b26cb3f6b39b1d7 [CANDIDATE]: Term 1 election: Requested vote from peers e44ce81184b94969854575ed79d06c48 (127.15.82.65:34763), bf412a3ffd4e41f5b81db3be3dfa6462 (127.15.82.67:36501)
I20260929 10:33:39.315665 23368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb730556d9f04d749be50a4aba18ab1e" candidate_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e44ce81184b94969854575ed79d06c48"
I20260929 10:33:39.315685 23648 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb730556d9f04d749be50a4aba18ab1e" candidate_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462"
I20260929 10:33:39.315797 23648 raft_consensus.cc:3037] T cb730556d9f04d749be50a4aba18ab1e P bf412a3ffd4e41f5b81db3be3dfa6462 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:33:39.315805 23368 raft_consensus.cc:3037] T cb730556d9f04d749be50a4aba18ab1e P e44ce81184b94969854575ed79d06c48 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:33:39.316828 23368 raft_consensus.cc:2454] T cb730556d9f04d749be50a4aba18ab1e P e44ce81184b94969854575ed79d06c48 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1f0b22653cc422d9b26cb3f6b39b1d7 in term 1.
I20260929 10:33:39.316897 23648 raft_consensus.cc:2454] T cb730556d9f04d749be50a4aba18ab1e P bf412a3ffd4e41f5b81db3be3dfa6462 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1f0b22653cc422d9b26cb3f6b39b1d7 in term 1.
I20260929 10:33:39.317133 23441 leader_election.cc:302] T cb730556d9f04d749be50a4aba18ab1e P c1f0b22653cc422d9b26cb3f6b39b1d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c1f0b22653cc422d9b26cb3f6b39b1d7, e44ce81184b94969854575ed79d06c48; no voters:
I20260929 10:33:39.317291 23730 raft_consensus.cc:2781] T cb730556d9f04d749be50a4aba18ab1e P c1f0b22653cc422d9b26cb3f6b39b1d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:33:39.317518 23730 raft_consensus.cc:686] T cb730556d9f04d749be50a4aba18ab1e P c1f0b22653cc422d9b26cb3f6b39b1d7 [term 1 LEADER]: Becoming Leader. State: Replica: c1f0b22653cc422d9b26cb3f6b39b1d7, State: Running, Role: LEADER
I20260929 10:33:39.317781 23730 consensus_queue.cc:237] T cb730556d9f04d749be50a4aba18ab1e P c1f0b22653cc422d9b26cb3f6b39b1d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44ce81184b94969854575ed79d06c48" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34763 } } peers { permanent_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34377 } } peers { permanent_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36501 } }
I20260929 10:33:39.320124 23209 catalog_manager.cc:5495] T cb730556d9f04d749be50a4aba18ab1e P c1f0b22653cc422d9b26cb3f6b39b1d7 reported cstate change: term changed from 0 to 1, leader changed from <none> to c1f0b22653cc422d9b26cb3f6b39b1d7 (127.15.82.66). New cstate: current_term: 1 leader_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44ce81184b94969854575ed79d06c48" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34763 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36501 } health_report { overall_health: UNKNOWN } } }
Sep 29 10:33:39 dist-test-slave-vg89 krb5kdc[23166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678017, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.82.126@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM
I20260929 10:33:39.408874 23761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:39.400264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:41463 (local address 127.0.0.1:36010)
0929 10:33:39.400639 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:39.400654 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:39.400770 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:39.401055 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:39.401058 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:39.401478 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:39.401583 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:39.401589 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:39.402331 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:39.402334 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:39.402966 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:39.402975 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:39.403397 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:39.403399 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:39.403459 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:39.404097 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:39.404110 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:39.405617 (+ 1507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:39.407396 (+ 1779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:39.407405 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:39.407410 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:39.407651 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:39.407939 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:39.407941 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:39.407942 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:39.408016 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:39.408377 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:39.408383 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:39.408485 (+ 102us) client_negotiation.cc:769] Sending connection context
0929 10:33:39.408576 (+ 91us) client_negotiation.cc:240] Negotiation successful
0929 10:33:39.408710 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":138,"threads_started":1}
I20260929 10:33:39.408905 23760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:39.400295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36010 (local address 127.15.82.126:41463)
0929 10:33:39.400571 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:39.400578 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:39.400733 (+ 155us) server_negotiation.cc:407] Connection header received
0929 10:33:39.400856 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:39.400858 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:39.400899 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:39.401001 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:39.401695 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:39.402236 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:39.403070 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:39.403295 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:39.403373 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:39.405740 (+ 2367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:39.405743 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:39.405746 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:39.405766 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:39.407292 (+ 1526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:39.407761 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:39.407764 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:39.407765 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:39.407850 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:39.408095 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:39.408098 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:39.408099 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:39.408287 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:39.408365 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:39.408636 (+ 271us) server_negotiation.cc:299] Negotiation successful
0929 10:33:39.408772 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":204,"thread_start_us":121,"threads_started":1}
I20260929 10:33:39.412720 23209 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.15.82.126@KRBTEST.COM'} at 127.0.0.1:36010:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260929 10:33:39.413487 23209 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:33:39.418562 23628 tablet_service.cc:1467] Processing CreateTablet for tablet 1b58540191e34a76a995df1f3e95c237 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7d820caa38484bd890ef6e3edd222154]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 10:33:39.418555 23348 tablet_service.cc:1467] Processing CreateTablet for tablet 1b58540191e34a76a995df1f3e95c237 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7d820caa38484bd890ef6e3edd222154]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 10:33:39.418560 23488 tablet_service.cc:1467] Processing CreateTablet for tablet 1b58540191e34a76a995df1f3e95c237 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7d820caa38484bd890ef6e3edd222154]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 10:33:39.418957 23628 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b58540191e34a76a995df1f3e95c237. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:33:39.418972 23348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b58540191e34a76a995df1f3e95c237. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:33:39.418969 23488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b58540191e34a76a995df1f3e95c237. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:33:39.421439 23724 tablet_bootstrap.cc:492] T 1b58540191e34a76a995df1f3e95c237 P c1f0b22653cc422d9b26cb3f6b39b1d7: Bootstrap starting.
I20260929 10:33:39.421887 23722 tablet_bootstrap.cc:492] T 1b58540191e34a76a995df1f3e95c237 P e44ce81184b94969854575ed79d06c48: Bootstrap starting.
I20260929 10:33:39.421950 23723 tablet_bootstrap.cc:492] T 1b58540191e34a76a995df1f3e95c237 P bf412a3ffd4e41f5b81db3be3dfa6462: Bootstrap starting.
I20260929 10:33:39.422514 23724 tablet_bootstrap.cc:654] T 1b58540191e34a76a995df1f3e95c237 P c1f0b22653cc422d9b26cb3f6b39b1d7: Neither blocks nor log segments found. Creating new log.
I20260929 10:33:39.422901 23722 tablet_bootstrap.cc:654] T 1b58540191e34a76a995df1f3e95c237 P e44ce81184b94969854575ed79d06c48: Neither blocks nor log segments found. Creating new log.
I20260929 10:33:39.422964 23723 tablet_bootstrap.cc:654] T 1b58540191e34a76a995df1f3e95c237 P bf412a3ffd4e41f5b81db3be3dfa6462: Neither blocks nor log segments found. Creating new log.
I20260929 10:33:39.423890 23724 tablet_bootstrap.cc:492] T 1b58540191e34a76a995df1f3e95c237 P c1f0b22653cc422d9b26cb3f6b39b1d7: No bootstrap required, opened a new log
I20260929 10:33:39.423987 23724 ts_tablet_manager.cc:1397] T 1b58540191e34a76a995df1f3e95c237 P c1f0b22653cc422d9b26cb3f6b39b1d7: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260929 10:33:39.424104 23722 tablet_bootstrap.cc:492] T 1b58540191e34a76a995df1f3e95c237 P e44ce81184b94969854575ed79d06c48: No bootstrap required, opened a new log
I20260929 10:33:39.424190 23722 ts_tablet_manager.cc:1397] T 1b58540191e34a76a995df1f3e95c237 P e44ce81184b94969854575ed79d06c48: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 10:33:39.424464 23724 raft_consensus.cc:348] T 1b58540191e34a76a995df1f3e95c237 P c1f0b22653cc422d9b26cb3f6b39b1d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44ce81184b94969854575ed79d06c48" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34763 } } peers { permanent_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34377 } } peers { permanent_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36501 } }
I20260929 10:33:39.424589 23724 raft_consensus.cc:374] T 1b58540191e34a76a995df1f3e95c237 P c1f0b22653cc422d9b26cb3f6b39b1d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:33:39.424629 23724 raft_consensus.cc:729] T 1b58540191e34a76a995df1f3e95c237 P c1f0b22653cc422d9b26cb3f6b39b1d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1f0b22653cc422d9b26cb3f6b39b1d7, State: Initialized, Role: FOLLOWER
I20260929 10:33:39.424625 23722 raft_consensus.cc:348] T 1b58540191e34a76a995df1f3e95c237 P e44ce81184b94969854575ed79d06c48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44ce81184b94969854575ed79d06c48" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34763 } } peers { permanent_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34377 } } peers { permanent_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36501 } }
I20260929 10:33:39.424733 23722 raft_consensus.cc:374] T 1b58540191e34a76a995df1f3e95c237 P e44ce81184b94969854575ed79d06c48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:33:39.424773 23722 raft_consensus.cc:729] T 1b58540191e34a76a995df1f3e95c237 P e44ce81184b94969854575ed79d06c48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e44ce81184b94969854575ed79d06c48, State: Initialized, Role: FOLLOWER
I20260929 10:33:39.424767 23724 consensus_queue.cc:260] T 1b58540191e34a76a995df1f3e95c237 P c1f0b22653cc422d9b26cb3f6b39b1d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44ce81184b94969854575ed79d06c48" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34763 } } peers { permanent_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34377 } } peers { permanent_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36501 } }
I20260929 10:33:39.424896 23722 consensus_queue.cc:260] T 1b58540191e34a76a995df1f3e95c237 P e44ce81184b94969854575ed79d06c48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44ce81184b94969854575ed79d06c48" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34763 } } peers { permanent_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34377 } } peers { permanent_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36501 } }
I20260929 10:33:39.425058 23724 ts_tablet_manager.cc:1428] T 1b58540191e34a76a995df1f3e95c237 P c1f0b22653cc422d9b26cb3f6b39b1d7: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260929 10:33:39.425168 23722 ts_tablet_manager.cc:1428] T 1b58540191e34a76a995df1f3e95c237 P e44ce81184b94969854575ed79d06c48: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260929 10:33:39.425143 23728 tablet_replica.cc:439] T 1b58540191e34a76a995df1f3e95c237 P e44ce81184b94969854575ed79d06c48: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44ce81184b94969854575ed79d06c48" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34763 } } peers { permanent_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34377 } } peers { permanent_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36501 } } }
I20260929 10:33:39.425287 23728 tablet_replica.cc:442] T 1b58540191e34a76a995df1f3e95c237 P e44ce81184b94969854575ed79d06c48: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 10:33:39.425334 23730 tablet_replica.cc:439] T 1b58540191e34a76a995df1f3e95c237 P c1f0b22653cc422d9b26cb3f6b39b1d7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44ce81184b94969854575ed79d06c48" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34763 } } peers { permanent_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34377 } } peers { permanent_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36501 } } }
I20260929 10:33:39.425467 23730 tablet_replica.cc:442] T 1b58540191e34a76a995df1f3e95c237 P c1f0b22653cc422d9b26cb3f6b39b1d7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 10:33:39.426505 23723 tablet_bootstrap.cc:492] T 1b58540191e34a76a995df1f3e95c237 P bf412a3ffd4e41f5b81db3be3dfa6462: No bootstrap required, opened a new log
I20260929 10:33:39.426620 23723 ts_tablet_manager.cc:1397] T 1b58540191e34a76a995df1f3e95c237 P bf412a3ffd4e41f5b81db3be3dfa6462: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260929 10:33:39.427073 23723 raft_consensus.cc:348] T 1b58540191e34a76a995df1f3e95c237 P bf412a3ffd4e41f5b81db3be3dfa6462 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44ce81184b94969854575ed79d06c48" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34763 } } peers { permanent_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34377 } } peers { permanent_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36501 } }
I20260929 10:33:39.427196 23723 raft_consensus.cc:374] T 1b58540191e34a76a995df1f3e95c237 P bf412a3ffd4e41f5b81db3be3dfa6462 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:33:39.427218 23723 raft_consensus.cc:729] T 1b58540191e34a76a995df1f3e95c237 P bf412a3ffd4e41f5b81db3be3dfa6462 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf412a3ffd4e41f5b81db3be3dfa6462, State: Initialized, Role: FOLLOWER
I20260929 10:33:39.427371 23723 consensus_queue.cc:260] T 1b58540191e34a76a995df1f3e95c237 P bf412a3ffd4e41f5b81db3be3dfa6462 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44ce81184b94969854575ed79d06c48" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34763 } } peers { permanent_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34377 } } peers { permanent_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36501 } }
I20260929 10:33:39.427695 23723 ts_tablet_manager.cc:1428] T 1b58540191e34a76a995df1f3e95c237 P bf412a3ffd4e41f5b81db3be3dfa6462: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260929 10:33:39.427659 23740 tablet_replica.cc:439] T 1b58540191e34a76a995df1f3e95c237 P bf412a3ffd4e41f5b81db3be3dfa6462: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44ce81184b94969854575ed79d06c48" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34763 } } peers { permanent_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34377 } } peers { permanent_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36501 } } }
I20260929 10:33:39.427816 23740 tablet_replica.cc:442] T 1b58540191e34a76a995df1f3e95c237 P bf412a3ffd4e41f5b81db3be3dfa6462: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 10:33:39.648128 23740 raft_consensus.cc:482] T 1b58540191e34a76a995df1f3e95c237 P bf412a3ffd4e41f5b81db3be3dfa6462 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:33:39.648295 23740 raft_consensus.cc:504] T 1b58540191e34a76a995df1f3e95c237 P bf412a3ffd4e41f5b81db3be3dfa6462 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44ce81184b94969854575ed79d06c48" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34763 } } peers { permanent_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34377 } } peers { permanent_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36501 } }
I20260929 10:33:39.648780 23740 leader_election.cc:288] T 1b58540191e34a76a995df1f3e95c237 P bf412a3ffd4e41f5b81db3be3dfa6462 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e44ce81184b94969854575ed79d06c48 (127.15.82.65:34763), c1f0b22653cc422d9b26cb3f6b39b1d7 (127.15.82.66:34377)
I20260929 10:33:39.649354 23505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b58540191e34a76a995df1f3e95c237" candidate_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" is_pre_election: true
I20260929 10:33:39.649354 23368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b58540191e34a76a995df1f3e95c237" candidate_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e44ce81184b94969854575ed79d06c48" is_pre_election: true
I20260929 10:33:39.649492 23505 raft_consensus.cc:2454] T 1b58540191e34a76a995df1f3e95c237 P c1f0b22653cc422d9b26cb3f6b39b1d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf412a3ffd4e41f5b81db3be3dfa6462 in term 0.
I20260929 10:33:39.649544 23368 raft_consensus.cc:2454] T 1b58540191e34a76a995df1f3e95c237 P e44ce81184b94969854575ed79d06c48 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf412a3ffd4e41f5b81db3be3dfa6462 in term 0.
I20260929 10:33:39.649852 23581 leader_election.cc:302] T 1b58540191e34a76a995df1f3e95c237 P bf412a3ffd4e41f5b81db3be3dfa6462 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bf412a3ffd4e41f5b81db3be3dfa6462, c1f0b22653cc422d9b26cb3f6b39b1d7; no voters:
I20260929 10:33:39.650036 23740 raft_consensus.cc:2781] T 1b58540191e34a76a995df1f3e95c237 P bf412a3ffd4e41f5b81db3be3dfa6462 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:33:39.650097 23740 raft_consensus.cc:482] T 1b58540191e34a76a995df1f3e95c237 P bf412a3ffd4e41f5b81db3be3dfa6462 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:33:39.650137 23740 raft_consensus.cc:3037] T 1b58540191e34a76a995df1f3e95c237 P bf412a3ffd4e41f5b81db3be3dfa6462 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:33:39.651232 23740 raft_consensus.cc:504] T 1b58540191e34a76a995df1f3e95c237 P bf412a3ffd4e41f5b81db3be3dfa6462 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44ce81184b94969854575ed79d06c48" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34763 } } peers { permanent_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34377 } } peers { permanent_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36501 } }
I20260929 10:33:39.651549 23740 leader_election.cc:288] T 1b58540191e34a76a995df1f3e95c237 P bf412a3ffd4e41f5b81db3be3dfa6462 [CANDIDATE]: Term 1 election: Requested vote from peers e44ce81184b94969854575ed79d06c48 (127.15.82.65:34763), c1f0b22653cc422d9b26cb3f6b39b1d7 (127.15.82.66:34377)
I20260929 10:33:39.651911 23368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b58540191e34a76a995df1f3e95c237" candidate_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e44ce81184b94969854575ed79d06c48"
I20260929 10:33:39.652014 23368 raft_consensus.cc:3037] T 1b58540191e34a76a995df1f3e95c237 P e44ce81184b94969854575ed79d06c48 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:33:39.651958 23505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b58540191e34a76a995df1f3e95c237" candidate_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7"
I20260929 10:33:39.652122 23505 raft_consensus.cc:3037] T 1b58540191e34a76a995df1f3e95c237 P c1f0b22653cc422d9b26cb3f6b39b1d7 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:33:39.652928 23368 raft_consensus.cc:2454] T 1b58540191e34a76a995df1f3e95c237 P e44ce81184b94969854575ed79d06c48 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf412a3ffd4e41f5b81db3be3dfa6462 in term 1.
I20260929 10:33:39.653097 23505 raft_consensus.cc:2454] T 1b58540191e34a76a995df1f3e95c237 P c1f0b22653cc422d9b26cb3f6b39b1d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf412a3ffd4e41f5b81db3be3dfa6462 in term 1.
I20260929 10:33:39.653250 23581 leader_election.cc:302] T 1b58540191e34a76a995df1f3e95c237 P bf412a3ffd4e41f5b81db3be3dfa6462 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bf412a3ffd4e41f5b81db3be3dfa6462, e44ce81184b94969854575ed79d06c48; no voters:
I20260929 10:33:39.653422 23740 raft_consensus.cc:2781] T 1b58540191e34a76a995df1f3e95c237 P bf412a3ffd4e41f5b81db3be3dfa6462 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:33:39.653612 23740 raft_consensus.cc:686] T 1b58540191e34a76a995df1f3e95c237 P bf412a3ffd4e41f5b81db3be3dfa6462 [term 1 LEADER]: Becoming Leader. State: Replica: bf412a3ffd4e41f5b81db3be3dfa6462, State: Running, Role: LEADER
I20260929 10:33:39.653792 23740 consensus_queue.cc:237] T 1b58540191e34a76a995df1f3e95c237 P bf412a3ffd4e41f5b81db3be3dfa6462 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44ce81184b94969854575ed79d06c48" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34763 } } peers { permanent_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34377 } } peers { permanent_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36501 } }
I20260929 10:33:39.654474 23765 tablet_replica.cc:439] T 1b58540191e34a76a995df1f3e95c237 P bf412a3ffd4e41f5b81db3be3dfa6462: TxnStatusTablet state changed. Reason: New leader bf412a3ffd4e41f5b81db3be3dfa6462. Latest consensus state: current_term: 1 leader_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44ce81184b94969854575ed79d06c48" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34763 } } peers { permanent_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34377 } } peers { permanent_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36501 } } }
I20260929 10:33:39.654595 23765 tablet_replica.cc:442] T 1b58540191e34a76a995df1f3e95c237 P bf412a3ffd4e41f5b81db3be3dfa6462: This TxnStatusTablet replica's current role is: LEADER
I20260929 10:33:39.654862 23767 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260929 10:33:39.655026 23767 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260929 10:33:39.655635 23209 catalog_manager.cc:5495] T 1b58540191e34a76a995df1f3e95c237 P bf412a3ffd4e41f5b81db3be3dfa6462 reported cstate change: term changed from 0 to 1, leader changed from <none> to bf412a3ffd4e41f5b81db3be3dfa6462 (127.15.82.67). New cstate: current_term: 1 leader_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44ce81184b94969854575ed79d06c48" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34763 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36501 } health_report { overall_health: HEALTHY } } }
I20260929 10:33:39.722167 23761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:39.718697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.67:36501 (local address 127.0.0.1:60176)
0929 10:33:39.718892 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:39.718909 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:39.719036 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:39.719350 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:39.719353 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:39.719360 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 10:33:39.719471 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:39.719476 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:39.720231 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:39.720236 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:39.720850 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:39.720858 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:39.721369 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:39.721372 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:39.721471 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:39.721520 (+ 49us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 10:33:39.721841 (+ 321us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 10:33:39.721861 (+ 20us) client_negotiation.cc:769] Sending connection context
0929 10:33:39.721929 (+ 68us) client_negotiation.cc:240] Negotiation successful
0929 10:33:39.722027 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260929 10:33:39.722167 23720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:39.718808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60176 (local address 127.15.82.67:36501)
0929 10:33:39.718929 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:39.718936 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:39.718957 (+ 21us) server_negotiation.cc:407] Connection header received
0929 10:33:39.719184 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:39.719187 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:39.719242 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:39.719328 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:33:39.719584 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:39.720114 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:39.721027 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:39.721248 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:39.721366 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:39.721665 (+ 299us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:33:39.721749 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:33:39.721834 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:39.721953 (+ 119us) server_negotiation.cc:299] Negotiation successful
0929 10:33:39.722008 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260929 10:33:39.725211 23505 raft_consensus.cc:1261] T 1b58540191e34a76a995df1f3e95c237 P c1f0b22653cc422d9b26cb3f6b39b1d7 [term 1 FOLLOWER]: Refusing update from remote peer bf412a3ffd4e41f5b81db3be3dfa6462: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:33:39.725306 23368 raft_consensus.cc:1261] T 1b58540191e34a76a995df1f3e95c237 P e44ce81184b94969854575ed79d06c48 [term 1 FOLLOWER]: Refusing update from remote peer bf412a3ffd4e41f5b81db3be3dfa6462: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:33:39.725728 23765 consensus_queue.cc:1035] T 1b58540191e34a76a995df1f3e95c237 P bf412a3ffd4e41f5b81db3be3dfa6462 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:33:39.725921 23740 consensus_queue.cc:1035] T 1b58540191e34a76a995df1f3e95c237 P bf412a3ffd4e41f5b81db3be3dfa6462 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e44ce81184b94969854575ed79d06c48" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:33:39.727360 23730 tablet_replica.cc:439] T 1b58540191e34a76a995df1f3e95c237 P c1f0b22653cc422d9b26cb3f6b39b1d7: TxnStatusTablet state changed. Reason: New leader bf412a3ffd4e41f5b81db3be3dfa6462. Latest consensus state: current_term: 1 leader_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44ce81184b94969854575ed79d06c48" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34763 } } peers { permanent_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34377 } } peers { permanent_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36501 } } }
I20260929 10:33:39.727474 23730 tablet_replica.cc:442] T 1b58540191e34a76a995df1f3e95c237 P c1f0b22653cc422d9b26cb3f6b39b1d7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 10:33:39.728369 23728 tablet_replica.cc:439] T 1b58540191e34a76a995df1f3e95c237 P e44ce81184b94969854575ed79d06c48: TxnStatusTablet state changed. Reason: New leader bf412a3ffd4e41f5b81db3be3dfa6462. Latest consensus state: current_term: 1 leader_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44ce81184b94969854575ed79d06c48" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34763 } } peers { permanent_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34377 } } peers { permanent_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36501 } } }
I20260929 10:33:39.728513 23728 tablet_replica.cc:442] T 1b58540191e34a76a995df1f3e95c237 P e44ce81184b94969854575ed79d06c48: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 10:33:39.730090 23770 mvcc.cc:205] Tried to move back new op lower bound from 7334617168789958656 to 7334617168503394304. Current Snapshot: MvccSnapshot[applied={T|T < 7334617168789958656}]
I20260929 10:33:39.730196 23730 tablet_replica.cc:439] T 1b58540191e34a76a995df1f3e95c237 P c1f0b22653cc422d9b26cb3f6b39b1d7: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44ce81184b94969854575ed79d06c48" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34763 } } peers { permanent_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34377 } } peers { permanent_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36501 } } }
I20260929 10:33:39.730302 23730 tablet_replica.cc:442] T 1b58540191e34a76a995df1f3e95c237 P c1f0b22653cc422d9b26cb3f6b39b1d7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 10:33:39.730264 23740 tablet_replica.cc:439] T 1b58540191e34a76a995df1f3e95c237 P bf412a3ffd4e41f5b81db3be3dfa6462: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44ce81184b94969854575ed79d06c48" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34763 } } peers { permanent_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34377 } } peers { permanent_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36501 } } }
I20260929 10:33:39.730366 23740 tablet_replica.cc:442] T 1b58540191e34a76a995df1f3e95c237 P bf412a3ffd4e41f5b81db3be3dfa6462: This TxnStatusTablet replica's current role is: LEADER
I20260929 10:33:39.730093 23765 tablet_replica.cc:439] T 1b58540191e34a76a995df1f3e95c237 P bf412a3ffd4e41f5b81db3be3dfa6462: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44ce81184b94969854575ed79d06c48" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34763 } } peers { permanent_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34377 } } peers { permanent_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36501 } } }
I20260929 10:33:39.731032 23765 tablet_replica.cc:442] T 1b58540191e34a76a995df1f3e95c237 P bf412a3ffd4e41f5b81db3be3dfa6462: This TxnStatusTablet replica's current role is: LEADER
I20260929 10:33:39.731294 23728 tablet_replica.cc:439] T 1b58540191e34a76a995df1f3e95c237 P e44ce81184b94969854575ed79d06c48: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44ce81184b94969854575ed79d06c48" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34763 } } peers { permanent_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34377 } } peers { permanent_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36501 } } }
I20260929 10:33:39.731402 23728 tablet_replica.cc:442] T 1b58540191e34a76a995df1f3e95c237 P e44ce81184b94969854575ed79d06c48: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 10:33:39.732154 23773 mvcc.cc:205] Tried to move back new op lower bound from 7334617168789958656 to 7334617168503394304. Current Snapshot: MvccSnapshot[applied={T|T < 7334617168789958656}]
I20260929 10:33:39.739308 23730 consensus_queue.cc:1035] T cb730556d9f04d749be50a4aba18ab1e P c1f0b22653cc422d9b26cb3f6b39b1d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e44ce81184b94969854575ed79d06c48" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:33:39.743966 23781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:39.738823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50580 (local address 127.15.82.66:34377)
0929 10:33:39.739056 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:39.739060 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:39.739072 (+ 12us) server_negotiation.cc:407] Connection header received
0929 10:33:39.739145 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:39.739147 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:39.739195 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:39.739327 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:33:39.739595 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:39.740685 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:39.741888 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:39.742142 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:39.742259 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:39.742617 (+ 358us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:33:39.743202 (+ 585us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:33:39.743724 (+ 522us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:39.743759 (+ 35us) server_negotiation.cc:299] Negotiation successful
0929 10:33:39.743815 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":73,"threads_started":1}
I20260929 10:33:39.746176 23730 consensus_queue.cc:1035] T cb730556d9f04d749be50a4aba18ab1e P c1f0b22653cc422d9b26cb3f6b39b1d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:33:39.755896 23787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:39.752256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.67:36501 (local address 127.15.82.66:59339)
0929 10:33:39.752555 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:39.752569 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:39.752657 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:39.752869 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:39.752871 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:39.752877 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 10:33:39.752982 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:39.752985 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:39.753906 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:39.753908 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:39.754644 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:39.754654 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:39.755253 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:39.755255 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:39.755324 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:39.755365 (+ 41us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 10:33:39.755656 (+ 291us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 10:33:39.755661 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 10:33:39.755710 (+ 49us) client_negotiation.cc:240] Negotiation successful
0929 10:33:39.755772 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":109,"threads_started":1}
I20260929 10:33:39.755918 23720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:39.752299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:59339 (local address 127.15.82.67:36501)
0929 10:33:39.752440 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:39.752445 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:39.752641 (+ 196us) server_negotiation.cc:407] Connection header received
0929 10:33:39.752728 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:39.752731 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:39.752779 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:39.752856 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:33:39.753080 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:39.753804 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:39.754756 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:39.755123 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:39.755256 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:39.755494 (+ 238us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:33:39.755573 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:33:39.755648 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:39.755745 (+ 97us) server_negotiation.cc:299] Negotiation successful
0929 10:33:39.755798 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 10:33:39.782164 23781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:39.777380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.67:52577 (local address 127.15.82.66:34377)
0929 10:33:39.777516 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:39.777523 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:39.777844 (+ 321us) server_negotiation.cc:407] Connection header received
0929 10:33:39.777947 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:39.777950 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:39.778005 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:39.778091 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:33:39.778489 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:39.779290 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:39.780669 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:39.780988 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:39.781127 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:39.781479 (+ 352us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:33:39.781563 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:33:39.781644 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:39.781927 (+ 283us) server_negotiation.cc:299] Negotiation successful
0929 10:33:39.781990 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260929 10:33:39.782642 23789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:39.777304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.66:34377 (local address 127.15.82.67:52577)
0929 10:33:39.777754 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:39.777766 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:39.777855 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:39.778209 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:39.778213 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:39.778222 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 10:33:39.778326 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:39.778332 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:39.779466 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:39.779470 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:39.780475 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:39.780486 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:39.781145 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:39.781150 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:39.781238 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:39.781299 (+ 61us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 10:33:39.781799 (+ 500us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 10:33:39.781806 (+ 7us) client_negotiation.cc:769] Sending connection context
0929 10:33:39.782416 (+ 610us) client_negotiation.cc:240] Negotiation successful
0929 10:33:39.782503 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":114,"threads_started":1}
I20260929 10:33:39.808159 23210 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36008:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:33:39.808349 23210 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36008:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:33:39.810777 23210 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f5a8c72fbf5a49f0a0dbf4031fe3ff93: Sending DeleteTablet for 3 replicas of tablet cb730556d9f04d749be50a4aba18ab1e
I20260929 10:33:39.811443 23628 tablet_service.cc:1514] Processing DeleteTablet for tablet cb730556d9f04d749be50a4aba18ab1e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:33:39 UTC) from {username='oryx', principal='oryx/127.15.82.126@KRBTEST.COM'} at 127.0.0.1:60164
I20260929 10:33:39.811610 23348 tablet_service.cc:1514] Processing DeleteTablet for tablet cb730556d9f04d749be50a4aba18ab1e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:33:39 UTC) from {username='oryx', principal='oryx/127.15.82.126@KRBTEST.COM'} at 127.0.0.1:40002
I20260929 10:33:39.812012 23793 tablet_replica.cc:330] T cb730556d9f04d749be50a4aba18ab1e P e44ce81184b94969854575ed79d06c48: stopping tablet replica
I20260929 10:33:39.812098 23488 tablet_service.cc:1514] Processing DeleteTablet for tablet cb730556d9f04d749be50a4aba18ab1e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:33:39 UTC) from {username='oryx', principal='oryx/127.15.82.126@KRBTEST.COM'} at 127.0.0.1:50570
I20260929 10:33:39.812155 23793 raft_consensus.cc:2229] T cb730556d9f04d749be50a4aba18ab1e P e44ce81184b94969854575ed79d06c48 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:33:39.812263 23793 raft_consensus.cc:2258] T cb730556d9f04d749be50a4aba18ab1e P e44ce81184b94969854575ed79d06c48 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:33:39.813043 23793 ts_tablet_manager.cc:1905] T cb730556d9f04d749be50a4aba18ab1e P e44ce81184b94969854575ed79d06c48: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:33:39.816835 23793 ts_tablet_manager.cc:1918] T cb730556d9f04d749be50a4aba18ab1e P e44ce81184b94969854575ed79d06c48: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 10:33:39.816912 23793 log.cc:1196] T cb730556d9f04d749be50a4aba18ab1e P e44ce81184b94969854575ed79d06c48: Deleting WAL directory at /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/ts-0/wal/wals/cb730556d9f04d749be50a4aba18ab1e
I20260929 10:33:39.817214 23793 ts_tablet_manager.cc:1939] T cb730556d9f04d749be50a4aba18ab1e P e44ce81184b94969854575ed79d06c48: Deleting consensus metadata
I20260929 10:33:39.817926 23197 catalog_manager.cc:4841] TS e44ce81184b94969854575ed79d06c48 (127.15.82.65:34763): tablet cb730556d9f04d749be50a4aba18ab1e (table test-table [id=7d2df19a126a4bbda5f01dfe4d014430]) successfully deleted
I20260929 10:33:39.818221 23794 tablet_replica.cc:330] T cb730556d9f04d749be50a4aba18ab1e P c1f0b22653cc422d9b26cb3f6b39b1d7: stopping tablet replica
I20260929 10:33:39.818572 23792 tablet_replica.cc:330] T cb730556d9f04d749be50a4aba18ab1e P bf412a3ffd4e41f5b81db3be3dfa6462: stopping tablet replica
I20260929 10:33:39.818748 23792 raft_consensus.cc:2229] T cb730556d9f04d749be50a4aba18ab1e P bf412a3ffd4e41f5b81db3be3dfa6462 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:33:39.818899 23792 raft_consensus.cc:2258] T cb730556d9f04d749be50a4aba18ab1e P bf412a3ffd4e41f5b81db3be3dfa6462 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:33:39.818363 23794 raft_consensus.cc:2229] T cb730556d9f04d749be50a4aba18ab1e P c1f0b22653cc422d9b26cb3f6b39b1d7 [term 1 LEADER]: Raft consensus shutting down.
I20260929 10:33:39.819550 23794 raft_consensus.cc:2258] T cb730556d9f04d749be50a4aba18ab1e P c1f0b22653cc422d9b26cb3f6b39b1d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:33:39.819687 23792 ts_tablet_manager.cc:1905] T cb730556d9f04d749be50a4aba18ab1e P bf412a3ffd4e41f5b81db3be3dfa6462: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:33:39.820230 23794 ts_tablet_manager.cc:1905] T cb730556d9f04d749be50a4aba18ab1e P c1f0b22653cc422d9b26cb3f6b39b1d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:33:39.822913 23792 ts_tablet_manager.cc:1918] T cb730556d9f04d749be50a4aba18ab1e P bf412a3ffd4e41f5b81db3be3dfa6462: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 10:33:39.822997 23794 ts_tablet_manager.cc:1918] T cb730556d9f04d749be50a4aba18ab1e P c1f0b22653cc422d9b26cb3f6b39b1d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 10:33:39.823074 23794 log.cc:1196] T cb730556d9f04d749be50a4aba18ab1e P c1f0b22653cc422d9b26cb3f6b39b1d7: Deleting WAL directory at /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/ts-1/wal/wals/cb730556d9f04d749be50a4aba18ab1e
I20260929 10:33:39.823185 23792 log.cc:1196] T cb730556d9f04d749be50a4aba18ab1e P bf412a3ffd4e41f5b81db3be3dfa6462: Deleting WAL directory at /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/ts-2/wal/wals/cb730556d9f04d749be50a4aba18ab1e
I20260929 10:33:39.823379 23794 ts_tablet_manager.cc:1939] T cb730556d9f04d749be50a4aba18ab1e P c1f0b22653cc422d9b26cb3f6b39b1d7: Deleting consensus metadata
I20260929 10:33:39.823643 23792 ts_tablet_manager.cc:1939] T cb730556d9f04d749be50a4aba18ab1e P bf412a3ffd4e41f5b81db3be3dfa6462: Deleting consensus metadata
I20260929 10:33:39.824291 23197 catalog_manager.cc:4841] TS c1f0b22653cc422d9b26cb3f6b39b1d7 (127.15.82.66:34377): tablet cb730556d9f04d749be50a4aba18ab1e (table test-table [id=7d2df19a126a4bbda5f01dfe4d014430]) successfully deleted
I20260929 10:33:39.824430 23196 catalog_manager.cc:4841] TS bf412a3ffd4e41f5b81db3be3dfa6462 (127.15.82.67:36501): tablet cb730556d9f04d749be50a4aba18ab1e (table test-table [id=7d2df19a126a4bbda5f01dfe4d014430]) successfully deleted
Sep 29 10:33:39 dist-test-slave-vg89 krb5kdc[23166](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678019, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 29 10:33:39 dist-test-slave-vg89 krb5kdc[23166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:33:39 dist-test-slave-vg89 krb5kdc[23166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
W20260929 10:33:39.846814 23760 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:33:39.839487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36014 (local address 127.15.82.126:41463)
0929 10:33:39.839670 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:39.839675 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:39.839847 (+ 172us) server_negotiation.cc:407] Connection header received
0929 10:33:39.839987 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:39.839990 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:39.840035 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:39.840109 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:39.841032 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:39.841565 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:39.842875 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:39.843103 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:39.843184 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:39.846684 (+ 3500us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36014: BlockingRecv error: recv got EOF from 127.0.0.1:36014 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Sep 29 10:33:39 dist-test-slave-vg89 krb5kdc[23166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678019, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM
I20260929 10:33:39.857633 23760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:39.848377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36020 (local address 127.15.82.126:41463)
0929 10:33:39.848496 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:39.848501 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:39.848747 (+ 246us) server_negotiation.cc:407] Connection header received
0929 10:33:39.848843 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:39.848846 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:39.848894 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:39.848982 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:39.849514 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:39.850380 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:39.851226 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:39.851583 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:39.851689 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:39.854167 (+ 2478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:39.854170 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:39.854172 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:39.854194 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:39.855956 (+ 1762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:39.856354 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:39.856362 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:39.856365 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:39.856457 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:39.856704 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:39.856707 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:39.856708 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:39.856931 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:39.857064 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:39.857328 (+ 264us) server_negotiation.cc:299] Negotiation successful
0929 10:33:39.857452 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 10:33:39.860436 23210 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36020:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 10:33:39.860911 23210 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:33:39.865559 23488 tablet_service.cc:1467] Processing CreateTablet for tablet dbd744e3c6264eb29311a08633e2a69d (DEFAULT_TABLE table=test-table [id=aeb106a06ed849a797f0038fd19b90e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:33:39.865568 23348 tablet_service.cc:1467] Processing CreateTablet for tablet dbd744e3c6264eb29311a08633e2a69d (DEFAULT_TABLE table=test-table [id=aeb106a06ed849a797f0038fd19b90e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:33:39.865870 23488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbd744e3c6264eb29311a08633e2a69d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:33:39.865873 23348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbd744e3c6264eb29311a08633e2a69d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:33:39.867821 23724 tablet_bootstrap.cc:492] T dbd744e3c6264eb29311a08633e2a69d P c1f0b22653cc422d9b26cb3f6b39b1d7: Bootstrap starting.
I20260929 10:33:39.868078 23628 tablet_service.cc:1467] Processing CreateTablet for tablet dbd744e3c6264eb29311a08633e2a69d (DEFAULT_TABLE table=test-table [id=aeb106a06ed849a797f0038fd19b90e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:33:39.868330 23722 tablet_bootstrap.cc:492] T dbd744e3c6264eb29311a08633e2a69d P e44ce81184b94969854575ed79d06c48: Bootstrap starting.
I20260929 10:33:39.868356 23628 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbd744e3c6264eb29311a08633e2a69d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:33:39.868778 23724 tablet_bootstrap.cc:654] T dbd744e3c6264eb29311a08633e2a69d P c1f0b22653cc422d9b26cb3f6b39b1d7: Neither blocks nor log segments found. Creating new log.
I20260929 10:33:39.869207 23722 tablet_bootstrap.cc:654] T dbd744e3c6264eb29311a08633e2a69d P e44ce81184b94969854575ed79d06c48: Neither blocks nor log segments found. Creating new log.
I20260929 10:33:39.870373 23722 tablet_bootstrap.cc:492] T dbd744e3c6264eb29311a08633e2a69d P e44ce81184b94969854575ed79d06c48: No bootstrap required, opened a new log
I20260929 10:33:39.870509 23722 ts_tablet_manager.cc:1397] T dbd744e3c6264eb29311a08633e2a69d P e44ce81184b94969854575ed79d06c48: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260929 10:33:39.870925 23723 tablet_bootstrap.cc:492] T dbd744e3c6264eb29311a08633e2a69d P bf412a3ffd4e41f5b81db3be3dfa6462: Bootstrap starting.
I20260929 10:33:39.870982 23722 raft_consensus.cc:348] T dbd744e3c6264eb29311a08633e2a69d P e44ce81184b94969854575ed79d06c48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44ce81184b94969854575ed79d06c48" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34763 } } peers { permanent_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34377 } } peers { permanent_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36501 } }
I20260929 10:33:39.871083 23722 raft_consensus.cc:374] T dbd744e3c6264eb29311a08633e2a69d P e44ce81184b94969854575ed79d06c48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:33:39.871114 23722 raft_consensus.cc:729] T dbd744e3c6264eb29311a08633e2a69d P e44ce81184b94969854575ed79d06c48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e44ce81184b94969854575ed79d06c48, State: Initialized, Role: FOLLOWER
I20260929 10:33:39.871234 23722 consensus_queue.cc:260] T dbd744e3c6264eb29311a08633e2a69d P e44ce81184b94969854575ed79d06c48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44ce81184b94969854575ed79d06c48" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34763 } } peers { permanent_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34377 } } peers { permanent_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36501 } }
I20260929 10:33:39.871510 23722 ts_tablet_manager.cc:1428] T dbd744e3c6264eb29311a08633e2a69d P e44ce81184b94969854575ed79d06c48: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260929 10:33:39.871717 23723 tablet_bootstrap.cc:654] T dbd744e3c6264eb29311a08633e2a69d P bf412a3ffd4e41f5b81db3be3dfa6462: Neither blocks nor log segments found. Creating new log.
I20260929 10:33:39.873029 23724 tablet_bootstrap.cc:492] T dbd744e3c6264eb29311a08633e2a69d P c1f0b22653cc422d9b26cb3f6b39b1d7: No bootstrap required, opened a new log
I20260929 10:33:39.873131 23724 ts_tablet_manager.cc:1397] T dbd744e3c6264eb29311a08633e2a69d P c1f0b22653cc422d9b26cb3f6b39b1d7: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20260929 10:33:39.873401 23723 tablet_bootstrap.cc:492] T dbd744e3c6264eb29311a08633e2a69d P bf412a3ffd4e41f5b81db3be3dfa6462: No bootstrap required, opened a new log
I20260929 10:33:39.873472 23723 ts_tablet_manager.cc:1397] T dbd744e3c6264eb29311a08633e2a69d P bf412a3ffd4e41f5b81db3be3dfa6462: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260929 10:33:39.873605 23724 raft_consensus.cc:348] T dbd744e3c6264eb29311a08633e2a69d P c1f0b22653cc422d9b26cb3f6b39b1d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44ce81184b94969854575ed79d06c48" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34763 } } peers { permanent_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34377 } } peers { permanent_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36501 } }
I20260929 10:33:39.873709 23724 raft_consensus.cc:374] T dbd744e3c6264eb29311a08633e2a69d P c1f0b22653cc422d9b26cb3f6b39b1d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:33:39.873742 23724 raft_consensus.cc:729] T dbd744e3c6264eb29311a08633e2a69d P c1f0b22653cc422d9b26cb3f6b39b1d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1f0b22653cc422d9b26cb3f6b39b1d7, State: Initialized, Role: FOLLOWER
I20260929 10:33:39.873863 23724 consensus_queue.cc:260] T dbd744e3c6264eb29311a08633e2a69d P c1f0b22653cc422d9b26cb3f6b39b1d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44ce81184b94969854575ed79d06c48" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34763 } } peers { permanent_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34377 } } peers { permanent_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36501 } }
I20260929 10:33:39.873867 23723 raft_consensus.cc:348] T dbd744e3c6264eb29311a08633e2a69d P bf412a3ffd4e41f5b81db3be3dfa6462 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44ce81184b94969854575ed79d06c48" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34763 } } peers { permanent_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34377 } } peers { permanent_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36501 } }
I20260929 10:33:39.873957 23723 raft_consensus.cc:374] T dbd744e3c6264eb29311a08633e2a69d P bf412a3ffd4e41f5b81db3be3dfa6462 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:33:39.873989 23723 raft_consensus.cc:729] T dbd744e3c6264eb29311a08633e2a69d P bf412a3ffd4e41f5b81db3be3dfa6462 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf412a3ffd4e41f5b81db3be3dfa6462, State: Initialized, Role: FOLLOWER
I20260929 10:33:39.874096 23723 consensus_queue.cc:260] T dbd744e3c6264eb29311a08633e2a69d P bf412a3ffd4e41f5b81db3be3dfa6462 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44ce81184b94969854575ed79d06c48" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34763 } } peers { permanent_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34377 } } peers { permanent_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36501 } }
I20260929 10:33:39.874157 23724 ts_tablet_manager.cc:1428] T dbd744e3c6264eb29311a08633e2a69d P c1f0b22653cc422d9b26cb3f6b39b1d7: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260929 10:33:39.874342 23723 ts_tablet_manager.cc:1428] T dbd744e3c6264eb29311a08633e2a69d P bf412a3ffd4e41f5b81db3be3dfa6462: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260929 10:33:39.921103 23765 raft_consensus.cc:482] T dbd744e3c6264eb29311a08633e2a69d P bf412a3ffd4e41f5b81db3be3dfa6462 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:33:39.921294 23765 raft_consensus.cc:504] T dbd744e3c6264eb29311a08633e2a69d P bf412a3ffd4e41f5b81db3be3dfa6462 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44ce81184b94969854575ed79d06c48" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34763 } } peers { permanent_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34377 } } peers { permanent_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36501 } }
I20260929 10:33:39.921733 23765 leader_election.cc:288] T dbd744e3c6264eb29311a08633e2a69d P bf412a3ffd4e41f5b81db3be3dfa6462 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e44ce81184b94969854575ed79d06c48 (127.15.82.65:34763), c1f0b22653cc422d9b26cb3f6b39b1d7 (127.15.82.66:34377)
I20260929 10:33:39.922310 23368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbd744e3c6264eb29311a08633e2a69d" candidate_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e44ce81184b94969854575ed79d06c48" is_pre_election: true
I20260929 10:33:39.922334 23505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbd744e3c6264eb29311a08633e2a69d" candidate_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" is_pre_election: true
I20260929 10:33:39.922501 23368 raft_consensus.cc:2454] T dbd744e3c6264eb29311a08633e2a69d P e44ce81184b94969854575ed79d06c48 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf412a3ffd4e41f5b81db3be3dfa6462 in term 0.
I20260929 10:33:39.922499 23505 raft_consensus.cc:2454] T dbd744e3c6264eb29311a08633e2a69d P c1f0b22653cc422d9b26cb3f6b39b1d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf412a3ffd4e41f5b81db3be3dfa6462 in term 0.
I20260929 10:33:39.922851 23581 leader_election.cc:302] T dbd744e3c6264eb29311a08633e2a69d P bf412a3ffd4e41f5b81db3be3dfa6462 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bf412a3ffd4e41f5b81db3be3dfa6462, c1f0b22653cc422d9b26cb3f6b39b1d7; no voters:
I20260929 10:33:39.923053 23765 raft_consensus.cc:2781] T dbd744e3c6264eb29311a08633e2a69d P bf412a3ffd4e41f5b81db3be3dfa6462 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:33:39.923111 23765 raft_consensus.cc:482] T dbd744e3c6264eb29311a08633e2a69d P bf412a3ffd4e41f5b81db3be3dfa6462 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:33:39.923166 23765 raft_consensus.cc:3037] T dbd744e3c6264eb29311a08633e2a69d P bf412a3ffd4e41f5b81db3be3dfa6462 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:33:39.924358 23765 raft_consensus.cc:504] T dbd744e3c6264eb29311a08633e2a69d P bf412a3ffd4e41f5b81db3be3dfa6462 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44ce81184b94969854575ed79d06c48" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34763 } } peers { permanent_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34377 } } peers { permanent_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36501 } }
I20260929 10:33:39.924724 23765 leader_election.cc:288] T dbd744e3c6264eb29311a08633e2a69d P bf412a3ffd4e41f5b81db3be3dfa6462 [CANDIDATE]: Term 1 election: Requested vote from peers e44ce81184b94969854575ed79d06c48 (127.15.82.65:34763), c1f0b22653cc422d9b26cb3f6b39b1d7 (127.15.82.66:34377)
I20260929 10:33:39.925086 23368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbd744e3c6264eb29311a08633e2a69d" candidate_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e44ce81184b94969854575ed79d06c48"
I20260929 10:33:39.925105 23505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbd744e3c6264eb29311a08633e2a69d" candidate_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7"
I20260929 10:33:39.925209 23505 raft_consensus.cc:3037] T dbd744e3c6264eb29311a08633e2a69d P c1f0b22653cc422d9b26cb3f6b39b1d7 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:33:39.925209 23368 raft_consensus.cc:3037] T dbd744e3c6264eb29311a08633e2a69d P e44ce81184b94969854575ed79d06c48 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:33:39.926294 23505 raft_consensus.cc:2454] T dbd744e3c6264eb29311a08633e2a69d P c1f0b22653cc422d9b26cb3f6b39b1d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf412a3ffd4e41f5b81db3be3dfa6462 in term 1.
I20260929 10:33:39.926294 23368 raft_consensus.cc:2454] T dbd744e3c6264eb29311a08633e2a69d P e44ce81184b94969854575ed79d06c48 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf412a3ffd4e41f5b81db3be3dfa6462 in term 1.
I20260929 10:33:39.926596 23581 leader_election.cc:302] T dbd744e3c6264eb29311a08633e2a69d P bf412a3ffd4e41f5b81db3be3dfa6462 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bf412a3ffd4e41f5b81db3be3dfa6462, c1f0b22653cc422d9b26cb3f6b39b1d7; no voters:
I20260929 10:33:39.926831 23765 raft_consensus.cc:2781] T dbd744e3c6264eb29311a08633e2a69d P bf412a3ffd4e41f5b81db3be3dfa6462 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:33:39.926944 23765 raft_consensus.cc:686] T dbd744e3c6264eb29311a08633e2a69d P bf412a3ffd4e41f5b81db3be3dfa6462 [term 1 LEADER]: Becoming Leader. State: Replica: bf412a3ffd4e41f5b81db3be3dfa6462, State: Running, Role: LEADER
I20260929 10:33:39.927093 23765 consensus_queue.cc:237] T dbd744e3c6264eb29311a08633e2a69d P bf412a3ffd4e41f5b81db3be3dfa6462 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44ce81184b94969854575ed79d06c48" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34763 } } peers { permanent_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34377 } } peers { permanent_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36501 } }
I20260929 10:33:39.928658 23209 catalog_manager.cc:5495] T dbd744e3c6264eb29311a08633e2a69d P bf412a3ffd4e41f5b81db3be3dfa6462 reported cstate change: term changed from 0 to 1, leader changed from <none> to bf412a3ffd4e41f5b81db3be3dfa6462 (127.15.82.67). New cstate: current_term: 1 leader_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44ce81184b94969854575ed79d06c48" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34763 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36501 } health_report { overall_health: HEALTHY } } }
I20260929 10:33:39.942576 23720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:39.939414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60182 (local address 127.15.82.67:36501)
0929 10:33:39.939542 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:39.939547 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:39.939561 (+ 14us) server_negotiation.cc:407] Connection header received
0929 10:33:39.939716 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:39.939719 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:39.939771 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:39.939849 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:33:39.940110 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:39.940621 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:39.941460 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:39.941692 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:39.941822 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:39.942100 (+ 278us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:33:39.942167 (+ 67us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:33:39.942243 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:39.942356 (+ 113us) server_negotiation.cc:299] Negotiation successful
0929 10:33:39.942435 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260929 10:33:39.944566 23368 raft_consensus.cc:1261] T dbd744e3c6264eb29311a08633e2a69d P e44ce81184b94969854575ed79d06c48 [term 1 FOLLOWER]: Refusing update from remote peer bf412a3ffd4e41f5b81db3be3dfa6462: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:33:39.944566 23505 raft_consensus.cc:1261] T dbd744e3c6264eb29311a08633e2a69d P c1f0b22653cc422d9b26cb3f6b39b1d7 [term 1 FOLLOWER]: Refusing update from remote peer bf412a3ffd4e41f5b81db3be3dfa6462: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:33:39.945024 23776 consensus_queue.cc:1035] T dbd744e3c6264eb29311a08633e2a69d P bf412a3ffd4e41f5b81db3be3dfa6462 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:33:39.945147 23765 consensus_queue.cc:1035] T dbd744e3c6264eb29311a08633e2a69d P bf412a3ffd4e41f5b81db3be3dfa6462 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e44ce81184b94969854575ed79d06c48" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:33:39.953066 23210 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36020:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:33:39.953182 23210 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36020:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:33:39.955431 23210 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f5a8c72fbf5a49f0a0dbf4031fe3ff93: Sending DeleteTablet for 3 replicas of tablet dbd744e3c6264eb29311a08633e2a69d
I20260929 10:33:39.956089 23488 tablet_service.cc:1514] Processing DeleteTablet for tablet dbd744e3c6264eb29311a08633e2a69d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:33:39 UTC) from {username='oryx', principal='oryx/127.15.82.126@KRBTEST.COM'} at 127.0.0.1:50570
I20260929 10:33:39.956236 23348 tablet_service.cc:1514] Processing DeleteTablet for tablet dbd744e3c6264eb29311a08633e2a69d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:33:39 UTC) from {username='oryx', principal='oryx/127.15.82.126@KRBTEST.COM'} at 127.0.0.1:40002
I20260929 10:33:39.956261 23628 tablet_service.cc:1514] Processing DeleteTablet for tablet dbd744e3c6264eb29311a08633e2a69d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:33:39 UTC) from {username='oryx', principal='oryx/127.15.82.126@KRBTEST.COM'} at 127.0.0.1:60164
I20260929 10:33:39.956408 23793 tablet_replica.cc:330] T dbd744e3c6264eb29311a08633e2a69d P e44ce81184b94969854575ed79d06c48: stopping tablet replica
I20260929 10:33:39.956421 23794 tablet_replica.cc:330] T dbd744e3c6264eb29311a08633e2a69d P c1f0b22653cc422d9b26cb3f6b39b1d7: stopping tablet replica
I20260929 10:33:39.956517 23793 raft_consensus.cc:2229] T dbd744e3c6264eb29311a08633e2a69d P e44ce81184b94969854575ed79d06c48 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:33:39.956540 23794 raft_consensus.cc:2229] T dbd744e3c6264eb29311a08633e2a69d P c1f0b22653cc422d9b26cb3f6b39b1d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:33:39.956645 23793 raft_consensus.cc:2258] T dbd744e3c6264eb29311a08633e2a69d P e44ce81184b94969854575ed79d06c48 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:33:39.956658 23794 raft_consensus.cc:2258] T dbd744e3c6264eb29311a08633e2a69d P c1f0b22653cc422d9b26cb3f6b39b1d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:33:39.956808 23792 tablet_replica.cc:330] T dbd744e3c6264eb29311a08633e2a69d P bf412a3ffd4e41f5b81db3be3dfa6462: stopping tablet replica
I20260929 10:33:39.956933 23792 raft_consensus.cc:2229] T dbd744e3c6264eb29311a08633e2a69d P bf412a3ffd4e41f5b81db3be3dfa6462 [term 1 LEADER]: Raft consensus shutting down.
I20260929 10:33:39.957119 23792 raft_consensus.cc:2258] T dbd744e3c6264eb29311a08633e2a69d P bf412a3ffd4e41f5b81db3be3dfa6462 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:33:39.957563 23210 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36020:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 10:33:39.957967 23794 ts_tablet_manager.cc:1905] T dbd744e3c6264eb29311a08633e2a69d P c1f0b22653cc422d9b26cb3f6b39b1d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:33:39.957966 23792 ts_tablet_manager.cc:1905] T dbd744e3c6264eb29311a08633e2a69d P bf412a3ffd4e41f5b81db3be3dfa6462: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260929 10:33:39.957964 23210 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:33:39.959545 23793 ts_tablet_manager.cc:1905] T dbd744e3c6264eb29311a08633e2a69d P e44ce81184b94969854575ed79d06c48: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:33:39.960357 23794 ts_tablet_manager.cc:1918] T dbd744e3c6264eb29311a08633e2a69d P c1f0b22653cc422d9b26cb3f6b39b1d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:33:39.960392 23792 ts_tablet_manager.cc:1918] T dbd744e3c6264eb29311a08633e2a69d P bf412a3ffd4e41f5b81db3be3dfa6462: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:33:39.960423 23794 log.cc:1196] T dbd744e3c6264eb29311a08633e2a69d P c1f0b22653cc422d9b26cb3f6b39b1d7: Deleting WAL directory at /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/ts-1/wal/wals/dbd744e3c6264eb29311a08633e2a69d
I20260929 10:33:39.960453 23792 log.cc:1196] T dbd744e3c6264eb29311a08633e2a69d P bf412a3ffd4e41f5b81db3be3dfa6462: Deleting WAL directory at /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/ts-2/wal/wals/dbd744e3c6264eb29311a08633e2a69d
I20260929 10:33:39.960690 23794 ts_tablet_manager.cc:1939] T dbd744e3c6264eb29311a08633e2a69d P c1f0b22653cc422d9b26cb3f6b39b1d7: Deleting consensus metadata
I20260929 10:33:39.960711 23792 ts_tablet_manager.cc:1939] T dbd744e3c6264eb29311a08633e2a69d P bf412a3ffd4e41f5b81db3be3dfa6462: Deleting consensus metadata
I20260929 10:33:39.961330 23196 catalog_manager.cc:4841] TS bf412a3ffd4e41f5b81db3be3dfa6462 (127.15.82.67:36501): tablet dbd744e3c6264eb29311a08633e2a69d (table test-table [id=aeb106a06ed849a797f0038fd19b90e4]) successfully deleted
I20260929 10:33:39.962174 23793 ts_tablet_manager.cc:1918] T dbd744e3c6264eb29311a08633e2a69d P e44ce81184b94969854575ed79d06c48: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:33:39.962270 23793 log.cc:1196] T dbd744e3c6264eb29311a08633e2a69d P e44ce81184b94969854575ed79d06c48: Deleting WAL directory at /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/ts-0/wal/wals/dbd744e3c6264eb29311a08633e2a69d
I20260929 10:33:39.962435 23197 catalog_manager.cc:4841] TS c1f0b22653cc422d9b26cb3f6b39b1d7 (127.15.82.66:34377): tablet dbd744e3c6264eb29311a08633e2a69d (table test-table [id=aeb106a06ed849a797f0038fd19b90e4]) successfully deleted
I20260929 10:33:39.962584 23793 ts_tablet_manager.cc:1939] T dbd744e3c6264eb29311a08633e2a69d P e44ce81184b94969854575ed79d06c48: Deleting consensus metadata
I20260929 10:33:39.963402 23197 catalog_manager.cc:4841] TS e44ce81184b94969854575ed79d06c48 (127.15.82.65:34763): tablet dbd744e3c6264eb29311a08633e2a69d (table test-table [id=aeb106a06ed849a797f0038fd19b90e4]) successfully deleted
I20260929 10:33:39.965160 23488 tablet_service.cc:1467] Processing CreateTablet for tablet 5cc67dc8e8b84dabb6a630a51b8f7527 (DEFAULT_TABLE table=test-table [id=b6ac73d7b20f4165a807e7c8361e1712]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:33:39.965248 23348 tablet_service.cc:1467] Processing CreateTablet for tablet 5cc67dc8e8b84dabb6a630a51b8f7527 (DEFAULT_TABLE table=test-table [id=b6ac73d7b20f4165a807e7c8361e1712]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:33:39.965332 23628 tablet_service.cc:1467] Processing CreateTablet for tablet 5cc67dc8e8b84dabb6a630a51b8f7527 (DEFAULT_TABLE table=test-table [id=b6ac73d7b20f4165a807e7c8361e1712]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:33:39.965402 23488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5cc67dc8e8b84dabb6a630a51b8f7527. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:33:39.965499 23348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5cc67dc8e8b84dabb6a630a51b8f7527. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:33:39.965552 23628 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5cc67dc8e8b84dabb6a630a51b8f7527. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:33:39.967640 23724 tablet_bootstrap.cc:492] T 5cc67dc8e8b84dabb6a630a51b8f7527 P c1f0b22653cc422d9b26cb3f6b39b1d7: Bootstrap starting.
I20260929 10:33:39.967732 23723 tablet_bootstrap.cc:492] T 5cc67dc8e8b84dabb6a630a51b8f7527 P bf412a3ffd4e41f5b81db3be3dfa6462: Bootstrap starting.
I20260929 10:33:39.968329 23723 tablet_bootstrap.cc:654] T 5cc67dc8e8b84dabb6a630a51b8f7527 P bf412a3ffd4e41f5b81db3be3dfa6462: Neither blocks nor log segments found. Creating new log.
I20260929 10:33:39.968495 23724 tablet_bootstrap.cc:654] T 5cc67dc8e8b84dabb6a630a51b8f7527 P c1f0b22653cc422d9b26cb3f6b39b1d7: Neither blocks nor log segments found. Creating new log.
I20260929 10:33:39.969249 23722 tablet_bootstrap.cc:492] T 5cc67dc8e8b84dabb6a630a51b8f7527 P e44ce81184b94969854575ed79d06c48: Bootstrap starting.
I20260929 10:33:39.969602 23723 tablet_bootstrap.cc:492] T 5cc67dc8e8b84dabb6a630a51b8f7527 P bf412a3ffd4e41f5b81db3be3dfa6462: No bootstrap required, opened a new log
I20260929 10:33:39.969682 23723 ts_tablet_manager.cc:1397] T 5cc67dc8e8b84dabb6a630a51b8f7527 P bf412a3ffd4e41f5b81db3be3dfa6462: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260929 10:33:39.969877 23724 tablet_bootstrap.cc:492] T 5cc67dc8e8b84dabb6a630a51b8f7527 P c1f0b22653cc422d9b26cb3f6b39b1d7: No bootstrap required, opened a new log
I20260929 10:33:39.969944 23724 ts_tablet_manager.cc:1397] T 5cc67dc8e8b84dabb6a630a51b8f7527 P c1f0b22653cc422d9b26cb3f6b39b1d7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260929 10:33:39.970134 23723 raft_consensus.cc:348] T 5cc67dc8e8b84dabb6a630a51b8f7527 P bf412a3ffd4e41f5b81db3be3dfa6462 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44ce81184b94969854575ed79d06c48" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34763 } } peers { permanent_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34377 } } peers { permanent_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36501 } }
I20260929 10:33:39.970237 23723 raft_consensus.cc:374] T 5cc67dc8e8b84dabb6a630a51b8f7527 P bf412a3ffd4e41f5b81db3be3dfa6462 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:33:39.970268 23723 raft_consensus.cc:729] T 5cc67dc8e8b84dabb6a630a51b8f7527 P bf412a3ffd4e41f5b81db3be3dfa6462 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf412a3ffd4e41f5b81db3be3dfa6462, State: Initialized, Role: FOLLOWER
I20260929 10:33:39.970335 23724 raft_consensus.cc:348] T 5cc67dc8e8b84dabb6a630a51b8f7527 P c1f0b22653cc422d9b26cb3f6b39b1d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44ce81184b94969854575ed79d06c48" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34763 } } peers { permanent_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34377 } } peers { permanent_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36501 } }
I20260929 10:33:39.970451 23724 raft_consensus.cc:374] T 5cc67dc8e8b84dabb6a630a51b8f7527 P c1f0b22653cc422d9b26cb3f6b39b1d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:33:39.970474 23722 tablet_bootstrap.cc:654] T 5cc67dc8e8b84dabb6a630a51b8f7527 P e44ce81184b94969854575ed79d06c48: Neither blocks nor log segments found. Creating new log.
I20260929 10:33:39.970481 23724 raft_consensus.cc:729] T 5cc67dc8e8b84dabb6a630a51b8f7527 P c1f0b22653cc422d9b26cb3f6b39b1d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1f0b22653cc422d9b26cb3f6b39b1d7, State: Initialized, Role: FOLLOWER
I20260929 10:33:39.970427 23723 consensus_queue.cc:260] T 5cc67dc8e8b84dabb6a630a51b8f7527 P bf412a3ffd4e41f5b81db3be3dfa6462 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44ce81184b94969854575ed79d06c48" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34763 } } peers { permanent_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34377 } } peers { permanent_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36501 } }
I20260929 10:33:39.970593 23724 consensus_queue.cc:260] T 5cc67dc8e8b84dabb6a630a51b8f7527 P c1f0b22653cc422d9b26cb3f6b39b1d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44ce81184b94969854575ed79d06c48" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34763 } } peers { permanent_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34377 } } peers { permanent_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36501 } }
I20260929 10:33:39.970717 23723 ts_tablet_manager.cc:1428] T 5cc67dc8e8b84dabb6a630a51b8f7527 P bf412a3ffd4e41f5b81db3be3dfa6462: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260929 10:33:39.970829 23724 ts_tablet_manager.cc:1428] T 5cc67dc8e8b84dabb6a630a51b8f7527 P c1f0b22653cc422d9b26cb3f6b39b1d7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260929 10:33:39.972076 23722 tablet_bootstrap.cc:492] T 5cc67dc8e8b84dabb6a630a51b8f7527 P e44ce81184b94969854575ed79d06c48: No bootstrap required, opened a new log
I20260929 10:33:39.972183 23722 ts_tablet_manager.cc:1397] T 5cc67dc8e8b84dabb6a630a51b8f7527 P e44ce81184b94969854575ed79d06c48: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260929 10:33:39.972656 23722 raft_consensus.cc:348] T 5cc67dc8e8b84dabb6a630a51b8f7527 P e44ce81184b94969854575ed79d06c48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44ce81184b94969854575ed79d06c48" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34763 } } peers { permanent_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34377 } } peers { permanent_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36501 } }
I20260929 10:33:39.972759 23722 raft_consensus.cc:374] T 5cc67dc8e8b84dabb6a630a51b8f7527 P e44ce81184b94969854575ed79d06c48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:33:39.972785 23722 raft_consensus.cc:729] T 5cc67dc8e8b84dabb6a630a51b8f7527 P e44ce81184b94969854575ed79d06c48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e44ce81184b94969854575ed79d06c48, State: Initialized, Role: FOLLOWER
I20260929 10:33:39.972884 23722 consensus_queue.cc:260] T 5cc67dc8e8b84dabb6a630a51b8f7527 P e44ce81184b94969854575ed79d06c48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44ce81184b94969854575ed79d06c48" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34763 } } peers { permanent_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34377 } } peers { permanent_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36501 } }
I20260929 10:33:39.973896 23722 ts_tablet_manager.cc:1428] T 5cc67dc8e8b84dabb6a630a51b8f7527 P e44ce81184b94969854575ed79d06c48: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260929 10:33:40.105485 23753 raft_consensus.cc:482] T 5cc67dc8e8b84dabb6a630a51b8f7527 P c1f0b22653cc422d9b26cb3f6b39b1d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:33:40.105628 23753 raft_consensus.cc:504] T 5cc67dc8e8b84dabb6a630a51b8f7527 P c1f0b22653cc422d9b26cb3f6b39b1d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44ce81184b94969854575ed79d06c48" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34763 } } peers { permanent_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34377 } } peers { permanent_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36501 } }
I20260929 10:33:40.106005 23753 leader_election.cc:288] T 5cc67dc8e8b84dabb6a630a51b8f7527 P c1f0b22653cc422d9b26cb3f6b39b1d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e44ce81184b94969854575ed79d06c48 (127.15.82.65:34763), bf412a3ffd4e41f5b81db3be3dfa6462 (127.15.82.67:36501)
I20260929 10:33:40.106500 23368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5cc67dc8e8b84dabb6a630a51b8f7527" candidate_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e44ce81184b94969854575ed79d06c48" is_pre_election: true
I20260929 10:33:40.106500 23648 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5cc67dc8e8b84dabb6a630a51b8f7527" candidate_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" is_pre_election: true
I20260929 10:33:40.106657 23368 raft_consensus.cc:2454] T 5cc67dc8e8b84dabb6a630a51b8f7527 P e44ce81184b94969854575ed79d06c48 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1f0b22653cc422d9b26cb3f6b39b1d7 in term 0.
I20260929 10:33:40.106657 23648 raft_consensus.cc:2454] T 5cc67dc8e8b84dabb6a630a51b8f7527 P bf412a3ffd4e41f5b81db3be3dfa6462 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1f0b22653cc422d9b26cb3f6b39b1d7 in term 0.
I20260929 10:33:40.106984 23441 leader_election.cc:302] T 5cc67dc8e8b84dabb6a630a51b8f7527 P c1f0b22653cc422d9b26cb3f6b39b1d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: bf412a3ffd4e41f5b81db3be3dfa6462, c1f0b22653cc422d9b26cb3f6b39b1d7, e44ce81184b94969854575ed79d06c48; no voters:
I20260929 10:33:40.107167 23753 raft_consensus.cc:2781] T 5cc67dc8e8b84dabb6a630a51b8f7527 P c1f0b22653cc422d9b26cb3f6b39b1d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:33:40.107229 23753 raft_consensus.cc:482] T 5cc67dc8e8b84dabb6a630a51b8f7527 P c1f0b22653cc422d9b26cb3f6b39b1d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:33:40.107266 23753 raft_consensus.cc:3037] T 5cc67dc8e8b84dabb6a630a51b8f7527 P c1f0b22653cc422d9b26cb3f6b39b1d7 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:33:40.108245 23753 raft_consensus.cc:504] T 5cc67dc8e8b84dabb6a630a51b8f7527 P c1f0b22653cc422d9b26cb3f6b39b1d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44ce81184b94969854575ed79d06c48" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34763 } } peers { permanent_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34377 } } peers { permanent_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36501 } }
I20260929 10:33:40.108579 23753 leader_election.cc:288] T 5cc67dc8e8b84dabb6a630a51b8f7527 P c1f0b22653cc422d9b26cb3f6b39b1d7 [CANDIDATE]: Term 1 election: Requested vote from peers e44ce81184b94969854575ed79d06c48 (127.15.82.65:34763), bf412a3ffd4e41f5b81db3be3dfa6462 (127.15.82.67:36501)
I20260929 10:33:40.108878 23368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5cc67dc8e8b84dabb6a630a51b8f7527" candidate_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e44ce81184b94969854575ed79d06c48"
I20260929 10:33:40.109036 23368 raft_consensus.cc:3037] T 5cc67dc8e8b84dabb6a630a51b8f7527 P e44ce81184b94969854575ed79d06c48 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:33:40.109001 23648 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5cc67dc8e8b84dabb6a630a51b8f7527" candidate_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462"
I20260929 10:33:40.109162 23648 raft_consensus.cc:3037] T 5cc67dc8e8b84dabb6a630a51b8f7527 P bf412a3ffd4e41f5b81db3be3dfa6462 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:33:40.110183 23368 raft_consensus.cc:2454] T 5cc67dc8e8b84dabb6a630a51b8f7527 P e44ce81184b94969854575ed79d06c48 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1f0b22653cc422d9b26cb3f6b39b1d7 in term 1.
I20260929 10:33:40.110311 23648 raft_consensus.cc:2454] T 5cc67dc8e8b84dabb6a630a51b8f7527 P bf412a3ffd4e41f5b81db3be3dfa6462 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1f0b22653cc422d9b26cb3f6b39b1d7 in term 1.
I20260929 10:33:40.110548 23441 leader_election.cc:302] T 5cc67dc8e8b84dabb6a630a51b8f7527 P c1f0b22653cc422d9b26cb3f6b39b1d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c1f0b22653cc422d9b26cb3f6b39b1d7, e44ce81184b94969854575ed79d06c48; no voters:
I20260929 10:33:40.110739 23753 raft_consensus.cc:2781] T 5cc67dc8e8b84dabb6a630a51b8f7527 P c1f0b22653cc422d9b26cb3f6b39b1d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:33:40.110858 23753 raft_consensus.cc:686] T 5cc67dc8e8b84dabb6a630a51b8f7527 P c1f0b22653cc422d9b26cb3f6b39b1d7 [term 1 LEADER]: Becoming Leader. State: Replica: c1f0b22653cc422d9b26cb3f6b39b1d7, State: Running, Role: LEADER
I20260929 10:33:40.110996 23753 consensus_queue.cc:237] T 5cc67dc8e8b84dabb6a630a51b8f7527 P c1f0b22653cc422d9b26cb3f6b39b1d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44ce81184b94969854575ed79d06c48" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34763 } } peers { permanent_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34377 } } peers { permanent_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36501 } }
I20260929 10:33:40.112677 23210 catalog_manager.cc:5495] T 5cc67dc8e8b84dabb6a630a51b8f7527 P c1f0b22653cc422d9b26cb3f6b39b1d7 reported cstate change: term changed from 0 to 1, leader changed from <none> to c1f0b22653cc422d9b26cb3f6b39b1d7 (127.15.82.66). New cstate: current_term: 1 leader_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44ce81184b94969854575ed79d06c48" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34763 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1f0b22653cc422d9b26cb3f6b39b1d7" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 34377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36501 } health_report { overall_health: UNKNOWN } } }
I20260929 10:33:40.141510 23781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:40.138814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50596 (local address 127.15.82.66:34377)
0929 10:33:40.138943 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:40.138947 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:40.138959 (+ 12us) server_negotiation.cc:407] Connection header received
0929 10:33:40.139060 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:40.139062 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:40.139102 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:40.139163 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:33:40.139361 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:40.139868 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:40.140627 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:40.140828 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:40.140901 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:40.141108 (+ 207us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:33:40.141171 (+ 63us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:33:40.141223 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:40.141302 (+ 79us) server_negotiation.cc:299] Negotiation successful
0929 10:33:40.141342 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260929 10:33:40.152864 23648 raft_consensus.cc:1261] T 5cc67dc8e8b84dabb6a630a51b8f7527 P bf412a3ffd4e41f5b81db3be3dfa6462 [term 1 FOLLOWER]: Refusing update from remote peer c1f0b22653cc422d9b26cb3f6b39b1d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:33:40.152916 23368 raft_consensus.cc:1261] T 5cc67dc8e8b84dabb6a630a51b8f7527 P e44ce81184b94969854575ed79d06c48 [term 1 FOLLOWER]: Refusing update from remote peer c1f0b22653cc422d9b26cb3f6b39b1d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:33:40.153281 23753 consensus_queue.cc:1035] T 5cc67dc8e8b84dabb6a630a51b8f7527 P c1f0b22653cc422d9b26cb3f6b39b1d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e44ce81184b94969854575ed79d06c48" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:33:40.153450 23783 consensus_queue.cc:1035] T 5cc67dc8e8b84dabb6a630a51b8f7527 P c1f0b22653cc422d9b26cb3f6b39b1d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf412a3ffd4e41f5b81db3be3dfa6462" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 36501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:33:40.182915 23209 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36020:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:33:40.183070 23209 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36020:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:33:40.184825 23209 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f5a8c72fbf5a49f0a0dbf4031fe3ff93: Sending DeleteTablet for 3 replicas of tablet 5cc67dc8e8b84dabb6a630a51b8f7527
I20260929 10:33:40.185362 23488 tablet_service.cc:1514] Processing DeleteTablet for tablet 5cc67dc8e8b84dabb6a630a51b8f7527 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:33:40 UTC) from {username='oryx', principal='oryx/127.15.82.126@KRBTEST.COM'} at 127.0.0.1:50570
I20260929 10:33:40.185369 23348 tablet_service.cc:1514] Processing DeleteTablet for tablet 5cc67dc8e8b84dabb6a630a51b8f7527 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:33:40 UTC) from {username='oryx', principal='oryx/127.15.82.126@KRBTEST.COM'} at 127.0.0.1:40002
I20260929 10:33:40.185557 23793 tablet_replica.cc:330] T 5cc67dc8e8b84dabb6a630a51b8f7527 P e44ce81184b94969854575ed79d06c48: stopping tablet replica
I20260929 10:33:40.185559 23794 tablet_replica.cc:330] T 5cc67dc8e8b84dabb6a630a51b8f7527 P c1f0b22653cc422d9b26cb3f6b39b1d7: stopping tablet replica
I20260929 10:33:40.185683 23793 raft_consensus.cc:2229] T 5cc67dc8e8b84dabb6a630a51b8f7527 P e44ce81184b94969854575ed79d06c48 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:33:40.185689 23794 raft_consensus.cc:2229] T 5cc67dc8e8b84dabb6a630a51b8f7527 P c1f0b22653cc422d9b26cb3f6b39b1d7 [term 1 LEADER]: Raft consensus shutting down.
I20260929 10:33:40.185708 23628 tablet_service.cc:1514] Processing DeleteTablet for tablet 5cc67dc8e8b84dabb6a630a51b8f7527 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:33:40 UTC) from {username='oryx', principal='oryx/127.15.82.126@KRBTEST.COM'} at 127.0.0.1:60164
I20260929 10:33:40.185798 23793 raft_consensus.cc:2258] T 5cc67dc8e8b84dabb6a630a51b8f7527 P e44ce81184b94969854575ed79d06c48 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:33:40.185858 23792 tablet_replica.cc:330] T 5cc67dc8e8b84dabb6a630a51b8f7527 P bf412a3ffd4e41f5b81db3be3dfa6462: stopping tablet replica
I20260929 10:33:40.185875 23794 raft_consensus.cc:2258] T 5cc67dc8e8b84dabb6a630a51b8f7527 P c1f0b22653cc422d9b26cb3f6b39b1d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:33:40.185973 23792 raft_consensus.cc:2229] T 5cc67dc8e8b84dabb6a630a51b8f7527 P bf412a3ffd4e41f5b81db3be3dfa6462 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:33:40.186088 23792 raft_consensus.cc:2258] T 5cc67dc8e8b84dabb6a630a51b8f7527 P bf412a3ffd4e41f5b81db3be3dfa6462 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:33:40.186192 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 23283
I20260929 10:33:40.186589 23794 ts_tablet_manager.cc:1905] T 5cc67dc8e8b84dabb6a630a51b8f7527 P c1f0b22653cc422d9b26cb3f6b39b1d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:33:40.187974 23792 ts_tablet_manager.cc:1905] T 5cc67dc8e8b84dabb6a630a51b8f7527 P bf412a3ffd4e41f5b81db3be3dfa6462: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:33:40.189719 23794 ts_tablet_manager.cc:1918] T 5cc67dc8e8b84dabb6a630a51b8f7527 P c1f0b22653cc422d9b26cb3f6b39b1d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 10:33:40.189807 23794 log.cc:1196] T 5cc67dc8e8b84dabb6a630a51b8f7527 P c1f0b22653cc422d9b26cb3f6b39b1d7: Deleting WAL directory at /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/ts-1/wal/wals/5cc67dc8e8b84dabb6a630a51b8f7527
I20260929 10:33:40.189986 23792 ts_tablet_manager.cc:1918] T 5cc67dc8e8b84dabb6a630a51b8f7527 P bf412a3ffd4e41f5b81db3be3dfa6462: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 10:33:40.190086 23792 log.cc:1196] T 5cc67dc8e8b84dabb6a630a51b8f7527 P bf412a3ffd4e41f5b81db3be3dfa6462: Deleting WAL directory at /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790677981592896-15689-0/minicluster-data/ts-2/wal/wals/5cc67dc8e8b84dabb6a630a51b8f7527
I20260929 10:33:40.190085 23794 ts_tablet_manager.cc:1939] T 5cc67dc8e8b84dabb6a630a51b8f7527 P c1f0b22653cc422d9b26cb3f6b39b1d7: Deleting consensus metadata
I20260929 10:33:40.190328 23792 ts_tablet_manager.cc:1939] T 5cc67dc8e8b84dabb6a630a51b8f7527 P bf412a3ffd4e41f5b81db3be3dfa6462: Deleting consensus metadata
I20260929 10:33:40.190837 23197 catalog_manager.cc:4841] TS c1f0b22653cc422d9b26cb3f6b39b1d7 (127.15.82.66:34377): tablet 5cc67dc8e8b84dabb6a630a51b8f7527 (table test-table [id=b6ac73d7b20f4165a807e7c8361e1712]) successfully deleted
I20260929 10:33:40.190970 23196 catalog_manager.cc:4841] TS bf412a3ffd4e41f5b81db3be3dfa6462 (127.15.82.67:36501): tablet 5cc67dc8e8b84dabb6a630a51b8f7527 (table test-table [id=b6ac73d7b20f4165a807e7c8361e1712]) successfully deleted
W20260929 10:33:40.194684 23197 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.82.65:34763 (error 108)
I20260929 10:33:40.194835 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 23423
I20260929 10:33:40.197161 23821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:40.196753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.65:34763 (local address 127.0.0.1:40018)
0929 10:33:40.196991 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:40.197077 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.65:34763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":98,"threads_started":1}
W20260929 10:33:40.197289 23197 catalog_manager.cc:4584] TS e44ce81184b94969854575ed79d06c48 (127.15.82.65:34763): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5cc67dc8e8b84dabb6a630a51b8f7527: Network error: Client connection negotiation failed: client connection to 127.15.82.65:34763: connect: Connection refused (error 111)
I20260929 10:33:40.201566 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 23563
I20260929 10:33:40.208766 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 23179
2026-09-29T10:33:40Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3876 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790677981592896-15689-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:33:40 dist-test-slave-vg89 krb5kdc[23824](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:33:40 dist-test-slave-vg89 krb5kdc[23824](info): set up 2 sockets
Sep 29 10:33:40 dist-test-slave-vg89 krb5kdc[23824](info): commencing operation
krb5kdc: starting...
W20260929 10:33:41.259341 15689 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Sep 29 10:33:41 dist-test-slave-vg89 krb5kdc[23824](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678021, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:33:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:33:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.82.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab.
Entry for principal kudu/127.15.82.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab.
I20260929 10:33:41.385504 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790677981592896-15689-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790677981592896-15689-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790677981592896-15689-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:45647
--webserver_interface=127.15.82.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab
--principal=kudu/127.15.82.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:43855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.82.126:45647
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:33:41.498596 23837 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:33:41.498910 23837 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:33:41.499006 23837 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:33:41.503306 23837 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:33:41.503388 23837 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:33:41.503430 23837 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:33:41.503468 23837 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:33:41.503502 23837 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:33:41.508800 23837 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:43855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790677981592896-15689-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790677981592896-15689-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.82.126:45647
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:45647
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.82.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23837
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790677981592896-15689-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:33:41.509910 23837 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:33:41.511025 23837 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:33:41.519891 23845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:41.519963 23837 server_base.cc:1034] running on GCE node
W20260929 10:33:41.519903 23842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:41.519901 23843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:41.520808 23837 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:33:41.521940 23837 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:33:41.523123 23837 hybrid_clock.cc:648] HybridClock initialized: now 1790678021523090 us; error 46 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-29T10:33:41Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1324 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:33:41 dist-test-slave-vg89 krb5kdc[23850](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:33:41 dist-test-slave-vg89 krb5kdc[23850](info): set up 2 sockets
Sep 29 10:33:41 dist-test-slave-vg89 krb5kdc[23850](info): commencing operation
krb5kdc: starting...
W20260929 10:33:42.571516 15689 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Sep 29 10:33:42 dist-test-slave-vg89 krb5kdc[23850](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678022, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:33:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:33:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.82.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab.
Entry for principal kudu/127.15.82.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab.
I20260929 10:33:42.696856 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:33713
--webserver_interface=127.15.82.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab
--principal=kudu/127.15.82.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:46833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.82.126:33713
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:33:42.807843 23863 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:33:42.808109 23863 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:33:42.808151 23863 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:33:42.812109 23863 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:33:42.812165 23863 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:33:42.812184 23863 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:33:42.812201 23863 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:33:42.812216 23863 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:33:42.817471 23863 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:46833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.82.126:33713
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab
--principal=kudu/127.15.82.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:33713
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.82.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23863
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:33:42.818553 23863 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:33:42.819577 23863 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:33:42.828119 23869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:42.828094 23868 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:42.828094 23871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:42.828274 23863 server_base.cc:1034] running on GCE node
I20260929 10:33:42.828948 23863 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:33:42.830089 23863 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:33:42.831278 23863 hybrid_clock.cc:648] HybridClock initialized: now 1790678022831266 us; error 29 us; skew 500 ppm
Sep 29 10:33:42 dist-test-slave-vg89 krb5kdc[23850](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678022, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:33:42.834021 23863 init.cc:377] Logged in from keytab as kudu/127.15.82.126@KRBTEST.COM (short username kudu)
I20260929 10:33:42.834987 23863 webserver.cc:460] Webserver started at http://127.15.82.126:38487/ using document root <none> and password file <none>
I20260929 10:33:42.835557 23863 fs_manager.cc:359] Metadata directory not provided
I20260929 10:33:42.835649 23863 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:33:42.835882 23863 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:33:42.837898 23863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/master-0/data/instance:
uuid: "c2b6f72a84fa476ab0ea7427249dcb75"
format_stamp: "Formatted at 2026-09-29 10:33:42 on dist-test-slave-vg89"
server_key: "16ad36b34d520f27c39dfcf009874700"
server_key_iv: "0c4560fb1fb7e9def014e6d5b1ff9d47"
server_key_version: "encryptionkey@0"
I20260929 10:33:42.838539 23863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/master-0/wal/instance:
uuid: "c2b6f72a84fa476ab0ea7427249dcb75"
format_stamp: "Formatted at 2026-09-29 10:33:42 on dist-test-slave-vg89"
server_key: "16ad36b34d520f27c39dfcf009874700"
server_key_iv: "0c4560fb1fb7e9def014e6d5b1ff9d47"
server_key_version: "encryptionkey@0"
I20260929 10:33:42.842209 23863 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260929 10:33:42.844712 23878 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:42.845860 23863 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 10:33:42.845999 23863 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/master-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/master-0/wal
uuid: "c2b6f72a84fa476ab0ea7427249dcb75"
format_stamp: "Formatted at 2026-09-29 10:33:42 on dist-test-slave-vg89"
server_key: "16ad36b34d520f27c39dfcf009874700"
server_key_iv: "0c4560fb1fb7e9def014e6d5b1ff9d47"
server_key_version: "encryptionkey@0"
I20260929 10:33:42.846122 23863 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:33:42.866237 23863 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:33:42.869254 23863 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:33:42.869459 23863 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:33:42.876556 23863 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.126:33713
I20260929 10:33:42.876575 23930 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.126:33713 every 8 connection(s)
I20260929 10:33:42.877476 23863 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
I20260929 10:33:42.880146 23931 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:33:42.883682 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 23863
I20260929 10:33:42.883829 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/master-0/wal/instance
I20260929 10:33:42.884104 15689 external_mini_cluster.cc:1428] Setting key 3c871c996778250de9b7d6da23ad6d2a
I20260929 10:33:42.885932 23931 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2b6f72a84fa476ab0ea7427249dcb75: Bootstrap starting.
I20260929 10:33:42.888420 23931 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c2b6f72a84fa476ab0ea7427249dcb75: Neither blocks nor log segments found. Creating new log.
I20260929 10:33:42.889138 23931 log.cc:824] T 00000000000000000000000000000000 P c2b6f72a84fa476ab0ea7427249dcb75: Log is configured to *not* fsync() on all Append() calls
I20260929 10:33:42.891330 23931 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2b6f72a84fa476ab0ea7427249dcb75: No bootstrap required, opened a new log
Sep 29 10:33:42 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678022, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:33:42.895893 23931 raft_consensus.cc:348] T 00000000000000000000000000000000 P c2b6f72a84fa476ab0ea7427249dcb75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2b6f72a84fa476ab0ea7427249dcb75" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 33713 } }
I20260929 10:33:42.896160 23931 raft_consensus.cc:374] T 00000000000000000000000000000000 P c2b6f72a84fa476ab0ea7427249dcb75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:33:42.896219 23931 raft_consensus.cc:729] T 00000000000000000000000000000000 P c2b6f72a84fa476ab0ea7427249dcb75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2b6f72a84fa476ab0ea7427249dcb75, State: Initialized, Role: FOLLOWER
I20260929 10:33:42.896700 23931 consensus_queue.cc:260] T 00000000000000000000000000000000 P c2b6f72a84fa476ab0ea7427249dcb75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2b6f72a84fa476ab0ea7427249dcb75" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 33713 } }
I20260929 10:33:42.896858 23931 raft_consensus.cc:388] T 00000000000000000000000000000000 P c2b6f72a84fa476ab0ea7427249dcb75 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:33:42.896922 23931 raft_consensus.cc:482] T 00000000000000000000000000000000 P c2b6f72a84fa476ab0ea7427249dcb75 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:33:42.897023 23931 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c2b6f72a84fa476ab0ea7427249dcb75 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:33:42.898142 23931 raft_consensus.cc:504] T 00000000000000000000000000000000 P c2b6f72a84fa476ab0ea7427249dcb75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2b6f72a84fa476ab0ea7427249dcb75" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 33713 } }
I20260929 10:33:42.898514 23931 leader_election.cc:302] T 00000000000000000000000000000000 P c2b6f72a84fa476ab0ea7427249dcb75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c2b6f72a84fa476ab0ea7427249dcb75; no voters:
I20260929 10:33:42.898823 23934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:42.885510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55218 (local address 127.15.82.126:33713)
0929 10:33:42.885919 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:42.885934 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:42.885968 (+ 34us) server_negotiation.cc:407] Connection header received
0929 10:33:42.886683 (+ 715us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:42.886701 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:42.887023 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:42.887305 (+ 282us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:42.888192 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:42.889221 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:42.890321 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:42.890838 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:42.890957 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:42.893653 (+ 2696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:42.893659 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:42.893675 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:42.893704 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:42.895660 (+ 1956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:42.896078 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:42.896085 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:42.896090 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:42.896231 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:42.896563 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:42.896566 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:42.896568 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:42.897073 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:42.897273 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:42.897527 (+ 254us) server_negotiation.cc:299] Negotiation successful
0929 10:33:42.897810 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":126,"threads_started":1}
I20260929 10:33:42.898828 23931 leader_election.cc:288] T 00000000000000000000000000000000 P c2b6f72a84fa476ab0ea7427249dcb75 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:33:42.898963 23936 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c2b6f72a84fa476ab0ea7427249dcb75 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:33:42.899183 23936 raft_consensus.cc:686] T 00000000000000000000000000000000 P c2b6f72a84fa476ab0ea7427249dcb75 [term 1 LEADER]: Becoming Leader. State: Replica: c2b6f72a84fa476ab0ea7427249dcb75, State: Running, Role: LEADER
I20260929 10:33:42.899549 23936 consensus_queue.cc:237] T 00000000000000000000000000000000 P c2b6f72a84fa476ab0ea7427249dcb75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2b6f72a84fa476ab0ea7427249dcb75" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 33713 } }
I20260929 10:33:42.900110 23931 sys_catalog.cc:564] T 00000000000000000000000000000000 P c2b6f72a84fa476ab0ea7427249dcb75 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:33:42.900758 23936 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2b6f72a84fa476ab0ea7427249dcb75 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c2b6f72a84fa476ab0ea7427249dcb75. Latest consensus state: current_term: 1 leader_uuid: "c2b6f72a84fa476ab0ea7427249dcb75" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2b6f72a84fa476ab0ea7427249dcb75" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 33713 } } }
I20260929 10:33:42.901490 23936 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2b6f72a84fa476ab0ea7427249dcb75 [sys.catalog]: This master's current role is: LEADER
I20260929 10:33:42.901382 23938 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2b6f72a84fa476ab0ea7427249dcb75 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c2b6f72a84fa476ab0ea7427249dcb75" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2b6f72a84fa476ab0ea7427249dcb75" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 33713 } } }
I20260929 10:33:42.901793 23938 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2b6f72a84fa476ab0ea7427249dcb75 [sys.catalog]: This master's current role is: LEADER
W20260929 10:33:42.904696 23951 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c2b6f72a84fa476ab0ea7427249dcb75: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 10:33:42.904783 23951 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 10:33:42.904851 23946 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:33:42.905665 23946 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:33:42.911284 23946 catalog_manager.cc:1347] Generated new cluster ID: 53c19fad2c594f8e854a4aa19df40d91
I20260929 10:33:42.911345 23946 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:33:42.925734 23946 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:33:42.927033 23946 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:33:42.933333 23946 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c2b6f72a84fa476ab0ea7427249dcb75: Generated new TSK 0
I20260929 10:33:42.934027 23946 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.82.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab.
Entry for principal kudu/127.15.82.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab.
I20260929 10:33:42.969419 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.65:0
--local_ip_for_outbound_sockets=127.15.82.65
--webserver_interface=127.15.82.65
--webserver_port=0
--tserver_master_addrs=127.15.82.126:33713
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab
--principal=kudu/127.15.82.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:46833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:33:43.086922 23957 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:33:43.087178 23957 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:33:43.087221 23957 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:33:43.091102 23957 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:33:43.091151 23957 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:33:43.091210 23957 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.65
I20260929 10:33:43.096277 23957 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:46833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab
--principal=kudu/127.15.82.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.82.65
--webserver_port=0
--tserver_master_addrs=127.15.82.126:33713
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23957
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.65
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:33:43.097180 23957 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:33:43.098093 23957 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:33:43.107606 23962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:43.107613 23963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:43.107678 23957 server_base.cc:1034] running on GCE node
W20260929 10:33:43.107641 23965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:43.108224 23957 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:33:43.108844 23957 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:33:43.110003 23957 hybrid_clock.cc:648] HybridClock initialized: now 1790678023109994 us; error 26 us; skew 500 ppm
Sep 29 10:33:43 dist-test-slave-vg89 krb5kdc[23850](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678023, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:33:43.112686 23957 init.cc:377] Logged in from keytab as kudu/127.15.82.65@KRBTEST.COM (short username kudu)
I20260929 10:33:43.113448 23957 webserver.cc:460] Webserver started at http://127.15.82.65:43383/ using document root <none> and password file <none>
I20260929 10:33:43.113956 23957 fs_manager.cc:359] Metadata directory not provided
I20260929 10:33:43.114012 23957 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:33:43.114168 23957 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:33:43.116163 23957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-0/data/instance:
uuid: "9a46938dd9cc47e1b1d4663caad005f4"
format_stamp: "Formatted at 2026-09-29 10:33:43 on dist-test-slave-vg89"
server_key: "9abae0053ed8573dd9ce9b731bee9b94"
server_key_iv: "1abb4ab0975e7524270c2aebff8ab468"
server_key_version: "encryptionkey@0"
I20260929 10:33:43.116673 23957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-0/wal/instance:
uuid: "9a46938dd9cc47e1b1d4663caad005f4"
format_stamp: "Formatted at 2026-09-29 10:33:43 on dist-test-slave-vg89"
server_key: "9abae0053ed8573dd9ce9b731bee9b94"
server_key_iv: "1abb4ab0975e7524270c2aebff8ab468"
server_key_version: "encryptionkey@0"
I20260929 10:33:43.120428 23957 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 10:33:43.122813 23972 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:43.124028 23957 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 10:33:43.124152 23957 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-0/wal
uuid: "9a46938dd9cc47e1b1d4663caad005f4"
format_stamp: "Formatted at 2026-09-29 10:33:43 on dist-test-slave-vg89"
server_key: "9abae0053ed8573dd9ce9b731bee9b94"
server_key_iv: "1abb4ab0975e7524270c2aebff8ab468"
server_key_version: "encryptionkey@0"
I20260929 10:33:43.124261 23957 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:33:43.144425 23957 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:33:43.147387 23957 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:33:43.147600 23957 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:33:43.148211 23957 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:33:43.149138 23957 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:33:43.149197 23957 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:43.149259 23957 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:33:43.149289 23957 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:43.158089 23957 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.65:37217
I20260929 10:33:43.158126 24085 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.65:37217 every 8 connection(s)
I20260929 10:33:43.158986 23957 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
I20260929 10:33:43.166916 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 23957
I20260929 10:33:43.167052 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-0/wal/instance
I20260929 10:33:43.167337 15689 external_mini_cluster.cc:1428] Setting key b090ca2f14f27d17f3e4b15931c4b1be
Sep 29 10:33:43 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678023, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.65@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:33:43.172362 23934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:43.160724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.65:55777 (local address 127.15.82.126:33713)
0929 10:33:43.160864 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:43.160868 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:43.161681 (+ 813us) server_negotiation.cc:407] Connection header received
0929 10:33:43.162979 (+ 1298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:43.162981 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:43.163026 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:43.163111 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:43.164182 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:43.164703 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:43.165760 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:43.165974 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:43.166055 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:43.168603 (+ 2548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:43.168606 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:43.168608 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:43.168630 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:43.170326 (+ 1696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:43.170865 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:43.170870 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:43.170871 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:43.170958 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:43.171307 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:43.171311 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:43.171313 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:43.171494 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:43.171596 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:43.172071 (+ 475us) server_negotiation.cc:299] Negotiation successful
0929 10:33:43.172196 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260929 10:33:43.173161 24088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:43.161074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:33713 (local address 127.15.82.65:55777)
0929 10:33:43.161512 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:43.161562 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:43.162801 (+ 1239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:43.163265 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:43.163271 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:43.163600 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:43.164058 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:43.164071 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:43.164803 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:43.164805 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:43.165661 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:43.165669 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:43.166094 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:43.166096 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:43.166226 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:43.166806 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:43.166821 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:43.168412 (+ 1591us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:43.170476 (+ 2064us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:43.170481 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:43.170491 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:43.170742 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:43.171054 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:43.171059 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:43.171061 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:43.171211 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:43.171605 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:43.171613 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:43.171838 (+ 225us) client_negotiation.cc:769] Sending connection context
0929 10:33:43.172021 (+ 183us) client_negotiation.cc:240] Negotiation successful
0929 10:33:43.172330 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":130,"threads_started":1}
I20260929 10:33:43.174322 24086 heartbeater.cc:347] Connected to a master server at 127.15.82.126:33713
I20260929 10:33:43.174623 24086 heartbeater.cc:464] Registering TS with master...
I20260929 10:33:43.175173 24086 heartbeater.cc:511] Master 127.15.82.126:33713 requested a full tablet report, sending...
I20260929 10:33:43.176826 23895 ts_manager.cc:194] Registered new tserver with Master: 9a46938dd9cc47e1b1d4663caad005f4 (127.15.82.65:37217)
WARNING: no policy specified for kudu/127.15.82.66@KRBTEST.COM; defaulting to no policy
I20260929 10:33:43.178174 23895 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.82.65@KRBTEST.COM'} at 127.15.82.65:55777
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab.
Entry for principal kudu/127.15.82.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab.
I20260929 10:33:43.195545 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.66:0
--local_ip_for_outbound_sockets=127.15.82.66
--webserver_interface=127.15.82.66
--webserver_port=0
--tserver_master_addrs=127.15.82.126:33713
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab
--principal=kudu/127.15.82.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:46833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:33:43.307145 24091 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:33:43.307385 24091 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:33:43.307427 24091 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:33:43.311406 24091 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:33:43.311456 24091 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:33:43.311511 24091 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.66
I20260929 10:33:43.316695 24091 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:46833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab
--principal=kudu/127.15.82.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.82.66
--webserver_port=0
--tserver_master_addrs=127.15.82.126:33713
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24091
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.66
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:33:43.317651 24091 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:33:43.318811 24091 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:33:43.328442 24099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:43.328610 24091 server_base.cc:1034] running on GCE node
W20260929 10:33:43.328449 24096 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:43.328454 24097 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:43.329120 24091 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:33:43.329738 24091 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:33:43.330905 24091 hybrid_clock.cc:648] HybridClock initialized: now 1790678023330883 us; error 35 us; skew 500 ppm
Sep 29 10:33:43 dist-test-slave-vg89 krb5kdc[23850](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678023, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:33:43.334038 24091 init.cc:377] Logged in from keytab as kudu/127.15.82.66@KRBTEST.COM (short username kudu)
I20260929 10:33:43.335000 24091 webserver.cc:460] Webserver started at http://127.15.82.66:45211/ using document root <none> and password file <none>
I20260929 10:33:43.335577 24091 fs_manager.cc:359] Metadata directory not provided
I20260929 10:33:43.335670 24091 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:33:43.335901 24091 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:33:43.338016 24091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-1/data/instance:
uuid: "b7fa32e5c9414ee2943a13c0551dd030"
format_stamp: "Formatted at 2026-09-29 10:33:43 on dist-test-slave-vg89"
server_key: "ad4b180e315bdf862933bc098da99ada"
server_key_iv: "4126ad2e0c56c4899b1fdadbb7e14e56"
server_key_version: "encryptionkey@0"
I20260929 10:33:43.338701 24091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-1/wal/instance:
uuid: "b7fa32e5c9414ee2943a13c0551dd030"
format_stamp: "Formatted at 2026-09-29 10:33:43 on dist-test-slave-vg89"
server_key: "ad4b180e315bdf862933bc098da99ada"
server_key_iv: "4126ad2e0c56c4899b1fdadbb7e14e56"
server_key_version: "encryptionkey@0"
I20260929 10:33:43.342391 24091 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260929 10:33:43.344938 24106 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:43.346158 24091 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 10:33:43.346294 24091 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-1/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-1/wal
uuid: "b7fa32e5c9414ee2943a13c0551dd030"
format_stamp: "Formatted at 2026-09-29 10:33:43 on dist-test-slave-vg89"
server_key: "ad4b180e315bdf862933bc098da99ada"
server_key_iv: "4126ad2e0c56c4899b1fdadbb7e14e56"
server_key_version: "encryptionkey@0"
I20260929 10:33:43.346403 24091 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:33:43.359771 24091 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:33:43.362594 24091 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:33:43.362789 24091 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:33:43.363363 24091 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:33:43.364164 24091 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:33:43.364234 24091 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:43.364307 24091 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:33:43.364349 24091 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:43.373311 24091 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.66:37575
I20260929 10:33:43.373332 24219 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.66:37575 every 8 connection(s)
I20260929 10:33:43.374199 24091 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
I20260929 10:33:43.382190 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 24091
I20260929 10:33:43.382335 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-1/wal/instance
I20260929 10:33:43.382652 15689 external_mini_cluster.cc:1428] Setting key 876132241b71f5ac03199623a783b0f0
Sep 29 10:33:43 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678023, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.66@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:33:43.387559 23934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:43.375965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:56783 (local address 127.15.82.126:33713)
0929 10:33:43.376111 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:43.376114 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:43.376915 (+ 801us) server_negotiation.cc:407] Connection header received
0929 10:33:43.378145 (+ 1230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:43.378147 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:43.378193 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:43.378268 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:43.379424 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:43.379970 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:43.381027 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:43.381243 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:43.381316 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:43.383700 (+ 2384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:43.383704 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:43.383707 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:43.383728 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:43.385220 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:43.385879 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:43.385885 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:43.385888 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:43.385987 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:43.386309 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:43.386314 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:43.386317 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:43.386594 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:43.386695 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:43.387168 (+ 473us) server_negotiation.cc:299] Negotiation successful
0929 10:33:43.387301 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260929 10:33:43.388199 24222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:43.376343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:33713 (local address 127.15.82.66:56783)
0929 10:33:43.376763 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:43.376803 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:43.377918 (+ 1115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:43.378466 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:43.378473 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:43.378815 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:43.379294 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:43.379308 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:43.380077 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:43.380079 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:43.380926 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:43.380934 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:43.381362 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:43.381364 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:43.381498 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:43.382018 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:43.382034 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:43.383551 (+ 1517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:43.385341 (+ 1790us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:43.385350 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:43.385368 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:43.385744 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:43.386084 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:43.386086 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:43.386088 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:43.386218 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:43.386703 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:43.386708 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:43.386926 (+ 218us) client_negotiation.cc:769] Sending connection context
0929 10:33:43.387091 (+ 165us) client_negotiation.cc:240] Negotiation successful
0929 10:33:43.387331 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":113,"threads_started":1}
I20260929 10:33:43.389418 24220 heartbeater.cc:347] Connected to a master server at 127.15.82.126:33713
I20260929 10:33:43.389701 24220 heartbeater.cc:464] Registering TS with master...
I20260929 10:33:43.390326 24220 heartbeater.cc:511] Master 127.15.82.126:33713 requested a full tablet report, sending...
I20260929 10:33:43.391793 23895 ts_manager.cc:194] Registered new tserver with Master: b7fa32e5c9414ee2943a13c0551dd030 (127.15.82.66:37575)
I20260929 10:33:43.392400 23895 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.82.66@KRBTEST.COM'} at 127.15.82.66:56783
WARNING: no policy specified for kudu/127.15.82.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab.
Entry for principal kudu/127.15.82.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab.
I20260929 10:33:43.412899 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.67:0
--local_ip_for_outbound_sockets=127.15.82.67
--webserver_interface=127.15.82.67
--webserver_port=0
--tserver_master_addrs=127.15.82.126:33713
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab
--principal=kudu/127.15.82.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:46833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:33:43.521993 24225 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:33:43.522259 24225 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:33:43.522348 24225 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:33:43.526566 24225 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:33:43.526625 24225 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:33:43.526741 24225 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.67
I20260929 10:33:43.532241 24225 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:46833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab
--principal=kudu/127.15.82.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.82.67
--webserver_port=0
--tserver_master_addrs=127.15.82.126:33713
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24225
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.67
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:33:43.533301 24225 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:33:43.534350 24225 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:33:43.543944 24231 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:43.543991 24230 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:43.543998 24233 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:43.544131 24225 server_base.cc:1034] running on GCE node
I20260929 10:33:43.544623 24225 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:33:43.545281 24225 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:33:43.546469 24225 hybrid_clock.cc:648] HybridClock initialized: now 1790678023546452 us; error 28 us; skew 500 ppm
Sep 29 10:33:43 dist-test-slave-vg89 krb5kdc[23850](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678023, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:33:43.549319 24225 init.cc:377] Logged in from keytab as kudu/127.15.82.67@KRBTEST.COM (short username kudu)
I20260929 10:33:43.550279 24225 webserver.cc:460] Webserver started at http://127.15.82.67:40465/ using document root <none> and password file <none>
I20260929 10:33:43.550897 24225 fs_manager.cc:359] Metadata directory not provided
I20260929 10:33:43.550993 24225 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:33:43.551205 24225 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:33:43.553215 24225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-2/data/instance:
uuid: "9be40c3c5ad1422e8221e7f285cb4d1e"
format_stamp: "Formatted at 2026-09-29 10:33:43 on dist-test-slave-vg89"
server_key: "043cf0e67b0ccddd4d745b7269511f15"
server_key_iv: "0ed236f73115e205f480059796164c7e"
server_key_version: "encryptionkey@0"
I20260929 10:33:43.553789 24225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-2/wal/instance:
uuid: "9be40c3c5ad1422e8221e7f285cb4d1e"
format_stamp: "Formatted at 2026-09-29 10:33:43 on dist-test-slave-vg89"
server_key: "043cf0e67b0ccddd4d745b7269511f15"
server_key_iv: "0ed236f73115e205f480059796164c7e"
server_key_version: "encryptionkey@0"
I20260929 10:33:43.557523 24225 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260929 10:33:43.560386 24240 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:43.562088 24225 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 10:33:43.562197 24225 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-2/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-2/wal
uuid: "9be40c3c5ad1422e8221e7f285cb4d1e"
format_stamp: "Formatted at 2026-09-29 10:33:43 on dist-test-slave-vg89"
server_key: "043cf0e67b0ccddd4d745b7269511f15"
server_key_iv: "0ed236f73115e205f480059796164c7e"
server_key_version: "encryptionkey@0"
I20260929 10:33:43.562297 24225 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:33:43.583606 24225 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:33:43.587368 24225 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:33:43.587550 24225 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:33:43.588124 24225 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:33:43.588959 24225 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:33:43.589027 24225 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:43.589069 24225 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:33:43.589092 24225 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:43.598572 24225 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.67:43203
I20260929 10:33:43.598632 24353 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.67:43203 every 8 connection(s)
I20260929 10:33:43.599576 24225 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
Sep 29 10:33:43 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678023, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.67@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:33:43.609724 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 24225
I20260929 10:33:43.609876 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-2/wal/instance
I20260929 10:33:43.610200 15689 external_mini_cluster.cc:1428] Setting key 2e16dacc5126e7f7675e7158437b353f
I20260929 10:33:43.613415 23934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:43.601499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.67:59407 (local address 127.15.82.126:33713)
0929 10:33:43.601678 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:43.601683 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:43.602673 (+ 990us) server_negotiation.cc:407] Connection header received
0929 10:33:43.603947 (+ 1274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:43.603950 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:43.604004 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:43.604086 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:43.605214 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:43.605803 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:43.607098 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:43.607316 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:43.607406 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:43.609705 (+ 2299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:43.609709 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:43.609712 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:43.609741 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:43.611348 (+ 1607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:43.611954 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:43.611956 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:43.611957 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:43.612031 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:43.612398 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:43.612401 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:43.612403 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:43.612649 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:43.612725 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:43.613183 (+ 458us) server_negotiation.cc:299] Negotiation successful
0929 10:33:43.613304 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260929 10:33:43.614289 24356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:43.602001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:33713 (local address 127.15.82.67:59407)
0929 10:33:43.602518 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:43.602563 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:43.603706 (+ 1143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:43.604247 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:43.604253 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:43.604603 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:43.605078 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:43.605091 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:43.605908 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:43.605911 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:43.606900 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:43.606909 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:43.607450 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:43.607452 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:43.607591 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:43.608129 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:43.608145 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:43.609534 (+ 1389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:43.611478 (+ 1944us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:43.611484 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:43.611499 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:43.611859 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:43.612120 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:33:43.612123 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:33:43.612124 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:33:43.612299 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:33:43.612754 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:43.612759 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:43.612978 (+ 219us) client_negotiation.cc:769] Sending connection context
0929 10:33:43.613139 (+ 161us) client_negotiation.cc:240] Negotiation successful
0929 10:33:43.613412 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":123,"threads_started":1}
I20260929 10:33:43.615432 24354 heartbeater.cc:347] Connected to a master server at 127.15.82.126:33713
I20260929 10:33:43.615671 24354 heartbeater.cc:464] Registering TS with master...
I20260929 10:33:43.616153 24354 heartbeater.cc:511] Master 127.15.82.126:33713 requested a full tablet report, sending...
I20260929 10:33:43.617105 23895 ts_manager.cc:194] Registered new tserver with Master: 9be40c3c5ad1422e8221e7f285cb4d1e (127.15.82.67:43203)
I20260929 10:33:43.617671 23895 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.82.67@KRBTEST.COM'} at 127.15.82.67:59407
I20260929 10:33:43.623965 15689 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.15.82.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.82.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.82.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/krb5kdc/oryx_127.15.82.65.keytab.
Entry for principal oryx/127.15.82.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/krb5kdc/oryx_127.15.82.65.keytab.
WARNING: no policy specified for oryx/127.15.82.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.82.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.82.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/krb5kdc/oryx_127.15.82.66.keytab.
Entry for principal oryx/127.15.82.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/krb5kdc/oryx_127.15.82.66.keytab.
WARNING: no policy specified for oryx/127.15.82.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.82.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.82.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/krb5kdc/oryx_127.15.82.67.keytab.
Entry for principal oryx/127.15.82.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/krb5kdc/oryx_127.15.82.67.keytab.
I20260929 10:33:43.697798 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 23957
I20260929 10:33:43.703675 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 24091
I20260929 10:33:43.709326 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 24225
I20260929 10:33:43.714794 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 23863
I20260929 10:33:43.721649 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:33713
--webserver_interface=127.15.82.126
--webserver_port=38487
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab
--principal=kudu/127.15.82.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:46833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.82.126:33713
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:33:43.831127 24363 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:33:43.831415 24363 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:33:43.831487 24363 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:33:43.835495 24363 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:33:43.835593 24363 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:33:43.835630 24363 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:33:43.835705 24363 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:33:43.835741 24363 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:33:43.840966 24363 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:46833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.82.126:33713
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab
--principal=kudu/127.15.82.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:33713
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.82.126
--webserver_port=38487
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24363
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:33:43.842027 24363 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:33:43.843107 24363 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:33:43.851446 24369 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:43.851410 24368 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:43.851504 24363 server_base.cc:1034] running on GCE node
W20260929 10:33:43.851442 24371 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:43.852217 24363 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:33:43.853266 24363 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:33:43.854472 24363 hybrid_clock.cc:648] HybridClock initialized: now 1790678023854445 us; error 25 us; skew 500 ppm
Sep 29 10:33:43 dist-test-slave-vg89 krb5kdc[23850](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678023, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:33:43.857216 24363 init.cc:377] Logged in from keytab as kudu/127.15.82.126@KRBTEST.COM (short username kudu)
I20260929 10:33:43.858091 24363 webserver.cc:460] Webserver started at http://127.15.82.126:38487/ using document root <none> and password file <none>
I20260929 10:33:43.858716 24363 fs_manager.cc:359] Metadata directory not provided
I20260929 10:33:43.858817 24363 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:33:43.862648 24363 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260929 10:33:43.864809 24378 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:43.866009 24363 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 10:33:43.866149 24363 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/master-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/master-0/wal
uuid: "c2b6f72a84fa476ab0ea7427249dcb75"
format_stamp: "Formatted at 2026-09-29 10:33:42 on dist-test-slave-vg89"
server_key: "16ad36b34d520f27c39dfcf009874700"
server_key_iv: "0c4560fb1fb7e9def014e6d5b1ff9d47"
server_key_version: "encryptionkey@0"
I20260929 10:33:43.866763 24363 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:33:43.876631 24363 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:33:43.879395 24363 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:33:43.879609 24363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:33:43.886461 24363 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.126:33713
I20260929 10:33:43.886508 24430 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.126:33713 every 8 connection(s)
I20260929 10:33:43.887485 24363 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
I20260929 10:33:43.888262 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 24363
I20260929 10:33:43.888784 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.65:37217
--local_ip_for_outbound_sockets=127.15.82.65
--tserver_master_addrs=127.15.82.126:33713
--webserver_port=43383
--webserver_interface=127.15.82.65
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab
--principal=kudu/127.15.82.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:46833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.15.82.65
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/krb5kdc/oryx_127.15.82.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 10:33:43.892669 24431 sys_catalog.cc:263] Verifying existing consensus state
I20260929 10:33:43.894246 24431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2b6f72a84fa476ab0ea7427249dcb75: Bootstrap starting.
I20260929 10:33:43.910573 24431 log.cc:824] T 00000000000000000000000000000000 P c2b6f72a84fa476ab0ea7427249dcb75: Log is configured to *not* fsync() on all Append() calls
I20260929 10:33:43.917068 24431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2b6f72a84fa476ab0ea7427249dcb75: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 10:33:43.917447 24431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2b6f72a84fa476ab0ea7427249dcb75: Bootstrap complete.
I20260929 10:33:43.921689 24431 raft_consensus.cc:348] T 00000000000000000000000000000000 P c2b6f72a84fa476ab0ea7427249dcb75 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2b6f72a84fa476ab0ea7427249dcb75" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 33713 } }
I20260929 10:33:43.922073 24431 raft_consensus.cc:729] T 00000000000000000000000000000000 P c2b6f72a84fa476ab0ea7427249dcb75 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2b6f72a84fa476ab0ea7427249dcb75, State: Initialized, Role: FOLLOWER
I20260929 10:33:43.922605 24431 consensus_queue.cc:260] T 00000000000000000000000000000000 P c2b6f72a84fa476ab0ea7427249dcb75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2b6f72a84fa476ab0ea7427249dcb75" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 33713 } }
I20260929 10:33:43.922732 24431 raft_consensus.cc:388] T 00000000000000000000000000000000 P c2b6f72a84fa476ab0ea7427249dcb75 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:33:43.922813 24431 raft_consensus.cc:482] T 00000000000000000000000000000000 P c2b6f72a84fa476ab0ea7427249dcb75 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:33:43.922907 24431 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c2b6f72a84fa476ab0ea7427249dcb75 [term 1 FOLLOWER]: Advancing to term 2
I20260929 10:33:43.924762 24431 raft_consensus.cc:504] T 00000000000000000000000000000000 P c2b6f72a84fa476ab0ea7427249dcb75 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2b6f72a84fa476ab0ea7427249dcb75" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 33713 } }
I20260929 10:33:43.925094 24431 leader_election.cc:302] T 00000000000000000000000000000000 P c2b6f72a84fa476ab0ea7427249dcb75 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c2b6f72a84fa476ab0ea7427249dcb75; no voters:
I20260929 10:33:43.925412 24431 leader_election.cc:288] T 00000000000000000000000000000000 P c2b6f72a84fa476ab0ea7427249dcb75 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260929 10:33:43.925505 24435 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c2b6f72a84fa476ab0ea7427249dcb75 [term 2 FOLLOWER]: Leader election won for term 2
I20260929 10:33:43.925695 24435 raft_consensus.cc:686] T 00000000000000000000000000000000 P c2b6f72a84fa476ab0ea7427249dcb75 [term 2 LEADER]: Becoming Leader. State: Replica: c2b6f72a84fa476ab0ea7427249dcb75, State: Running, Role: LEADER
I20260929 10:33:43.925985 24435 consensus_queue.cc:237] T 00000000000000000000000000000000 P c2b6f72a84fa476ab0ea7427249dcb75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2b6f72a84fa476ab0ea7427249dcb75" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 33713 } }
I20260929 10:33:43.926635 24431 sys_catalog.cc:564] T 00000000000000000000000000000000 P c2b6f72a84fa476ab0ea7427249dcb75 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:33:43.926939 24435 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2b6f72a84fa476ab0ea7427249dcb75 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c2b6f72a84fa476ab0ea7427249dcb75. Latest consensus state: current_term: 2 leader_uuid: "c2b6f72a84fa476ab0ea7427249dcb75" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2b6f72a84fa476ab0ea7427249dcb75" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 33713 } } }
I20260929 10:33:43.927054 24435 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2b6f72a84fa476ab0ea7427249dcb75 [sys.catalog]: This master's current role is: LEADER
I20260929 10:33:43.927464 24436 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2b6f72a84fa476ab0ea7427249dcb75 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "c2b6f72a84fa476ab0ea7427249dcb75" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2b6f72a84fa476ab0ea7427249dcb75" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 33713 } } }
I20260929 10:33:43.927649 24436 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2b6f72a84fa476ab0ea7427249dcb75 [sys.catalog]: This master's current role is: LEADER
I20260929 10:33:43.928611 24439 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:33:43.932001 24439 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:33:43.932641 24439 catalog_manager.cc:1259] Loaded cluster ID: 53c19fad2c594f8e854a4aa19df40d91
I20260929 10:33:43.932700 24439 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:33:43.935400 24439 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:33:43.936054 24439 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c2b6f72a84fa476ab0ea7427249dcb75: Loaded TSK: 0
I20260929 10:33:43.937248 24439 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260929 10:33:44.003358 24433 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:33:44.003620 24433 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:33:44.003662 24433 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:33:44.007838 24433 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:33:44.007897 24433 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:33:44.007956 24433 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.65
I20260929 10:33:44.013165 24433 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:46833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/krb5kdc/oryx_127.15.82.65.keytab
--principal=oryx/127.15.82.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.65:37217
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.82.65
--webserver_port=43383
--tserver_master_addrs=127.15.82.126:33713
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24433
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.65
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:33:44.014223 24433 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:33:44.015298 24433 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:33:44.024870 24457 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:44.024962 24458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:44.025257 24433 server_base.cc:1034] running on GCE node
W20260929 10:33:44.025309 24460 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:44.025765 24433 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:33:44.026634 24433 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:33:44.027823 24433 hybrid_clock.cc:648] HybridClock initialized: now 1790678024027795 us; error 49 us; skew 500 ppm
Sep 29 10:33:44 dist-test-slave-vg89 krb5kdc[23850](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678024, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.82.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:33:44.031126 24433 init.cc:377] Logged in from keytab as oryx/127.15.82.65@KRBTEST.COM (short username oryx)
I20260929 10:33:44.032056 24433 webserver.cc:460] Webserver started at http://127.15.82.65:43383/ using document root <none> and password file <none>
I20260929 10:33:44.032653 24433 fs_manager.cc:359] Metadata directory not provided
I20260929 10:33:44.032727 24433 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:33:44.036712 24433 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260929 10:33:44.038933 24467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:44.040299 24433 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 10:33:44.040441 24433 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-0/wal
uuid: "9a46938dd9cc47e1b1d4663caad005f4"
format_stamp: "Formatted at 2026-09-29 10:33:43 on dist-test-slave-vg89"
server_key: "9abae0053ed8573dd9ce9b731bee9b94"
server_key_iv: "1abb4ab0975e7524270c2aebff8ab468"
server_key_version: "encryptionkey@0"
I20260929 10:33:44.041002 24433 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:33:44.055495 24433 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:33:44.058956 24433 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:33:44.059176 24433 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:33:44.059803 24433 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:33:44.060698 24433 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:33:44.060777 24433 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:44.060853 24433 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:33:44.060889 24433 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:44.071612 24433 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.65:37217
I20260929 10:33:44.071635 24580 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.65:37217 every 8 connection(s)
I20260929 10:33:44.072677 24433 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
I20260929 10:33:44.075661 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 24433
I20260929 10:33:44.076233 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.66:37575
--local_ip_for_outbound_sockets=127.15.82.66
--tserver_master_addrs=127.15.82.126:33713
--webserver_port=45211
--webserver_interface=127.15.82.66
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab
--principal=kudu/127.15.82.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:46833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.15.82.66
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/krb5kdc/oryx_127.15.82.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 10:33:44 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.65@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:33:44 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.65@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
W20260929 10:33:44.086547 24583 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:33:44.074710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.65:58293 (local address 127.15.82.126:33713)
0929 10:33:44.075177 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:44.075198 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:44.075617 (+ 419us) server_negotiation.cc:407] Connection header received
0929 10:33:44.077695 (+ 2078us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:44.077706 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:44.078010 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:44.078349 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:44.079561 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:44.080540 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:44.081678 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:44.082149 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:44.082332 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:44.085769 (+ 3437us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.82.65:58293: BlockingRecv error: recv got EOF from 127.15.82.65:58293 (error 108)
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":92,"threads_started":1}
I20260929 10:33:44.086558 24584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:44.074862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:33713 (local address 127.15.82.65:58293)
0929 10:33:44.075398 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:44.075462 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:44.077276 (+ 1814us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:44.078457 (+ 1181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:44.078465 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:44.078889 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:44.079354 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:44.079367 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:44.080681 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:44.080685 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:44.081547 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:44.081556 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:44.082320 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:44.082324 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:44.082546 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:44.083172 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:44.083189 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:44.085548 (+ 2359us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.82.126:33713: Server oryx/127.15.82.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":152,"threads_started":1}
W20260929 10:33:44.087030 24581 heartbeater.cc:644] Failed to heartbeat to 127.15.82.126:33713 (0 consecutive failures): Not authorized: Failed to ping master at 127.15.82.126:33713: Client connection negotiation failed: client connection to 127.15.82.126:33713: Server oryx/127.15.82.126@KRBTEST.COM not found in Kerberos database
Sep 29 10:33:44 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.65@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:33:44 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.65@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:33:44.092859 24584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:44.087499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:33713 (local address 127.15.82.65:54293)
0929 10:33:44.087624 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:44.087640 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:44.087719 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:44.087934 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:44.087936 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:44.088064 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:44.088151 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:44.088155 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:44.089172 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:44.089174 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:44.089704 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:44.089711 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:44.090273 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:44.090275 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:44.090369 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:44.090890 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:44.090901 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:44.092723 (+ 1822us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.82.126:33713: Server oryx/127.15.82.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":48}
W20260929 10:33:44.092952 24583 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:33:44.087596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.65:54293 (local address 127.15.82.126:33713)
0929 10:33:44.087713 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:44.087718 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:44.087733 (+ 15us) server_negotiation.cc:407] Connection header received
0929 10:33:44.087779 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:44.087782 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:44.087829 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:44.087913 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:44.088252 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:44.089072 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:44.089805 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:44.090172 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:44.090271 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:44.092835 (+ 2564us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.82.65:54293: BlockingRecv error: recv got EOF from 127.15.82.65:54293 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Sep 29 10:33:44 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.65@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:33:44 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.65@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:33:44.098253 24584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:44.093346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:33713 (local address 127.15.82.65:55753)
0929 10:33:44.093463 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:44.093480 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:44.093564 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:44.093767 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:44.093769 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:44.093885 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:44.093951 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:44.093955 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:44.094968 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:44.094970 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:44.095461 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:44.095468 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:44.096023 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:44.096025 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:44.096080 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:44.096549 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:44.096560 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:44.098163 (+ 1603us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.82.126:33713: Server oryx/127.15.82.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":48}
W20260929 10:33:44.098454 24583 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:33:44.093461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.65:55753 (local address 127.15.82.126:33713)
0929 10:33:44.093572 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:44.093576 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:44.093592 (+ 16us) server_negotiation.cc:407] Connection header received
0929 10:33:44.093626 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:44.093629 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:44.093670 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:44.093747 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:44.094054 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:44.094858 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:44.095577 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:44.095918 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:44.096027 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:44.098313 (+ 2286us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.82.65:55753: BlockingRecv error: recv got EOF from 127.15.82.65:55753 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260929 10:33:44.098551 24581 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 10:33:44.191344 24585 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:33:44.191689 24585 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:33:44.191768 24585 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:33:44.195757 24585 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:33:44.195855 24585 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:33:44.195950 24585 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.66
I20260929 10:33:44.201143 24585 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:46833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/krb5kdc/oryx_127.15.82.66.keytab
--principal=oryx/127.15.82.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.66:37575
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.82.66
--webserver_port=45211
--tserver_master_addrs=127.15.82.126:33713
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24585
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.66
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:33:44.202245 24585 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:33:44.203356 24585 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:33:44.213595 24593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:44.213596 24591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:44.213644 24585 server_base.cc:1034] running on GCE node
W20260929 10:33:44.213621 24590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:44.214202 24585 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:33:44.214900 24585 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:33:44.216084 24585 hybrid_clock.cc:648] HybridClock initialized: now 1790678024216068 us; error 28 us; skew 500 ppm
Sep 29 10:33:44 dist-test-slave-vg89 krb5kdc[23850](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678024, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.82.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:33:44.218686 24585 init.cc:377] Logged in from keytab as oryx/127.15.82.66@KRBTEST.COM (short username oryx)
I20260929 10:33:44.219499 24585 webserver.cc:460] Webserver started at http://127.15.82.66:45211/ using document root <none> and password file <none>
I20260929 10:33:44.220067 24585 fs_manager.cc:359] Metadata directory not provided
I20260929 10:33:44.220171 24585 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:33:44.224094 24585 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.002s
I20260929 10:33:44.226074 24600 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:44.227455 24585 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 10:33:44.227587 24585 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-1/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-1/wal
uuid: "b7fa32e5c9414ee2943a13c0551dd030"
format_stamp: "Formatted at 2026-09-29 10:33:43 on dist-test-slave-vg89"
server_key: "ad4b180e315bdf862933bc098da99ada"
server_key_iv: "4126ad2e0c56c4899b1fdadbb7e14e56"
server_key_version: "encryptionkey@0"
I20260929 10:33:44.228106 24585 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:33:44.243777 24585 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:33:44.246806 24585 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:33:44.246956 24585 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:33:44.247489 24585 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:33:44.248296 24585 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:33:44.248342 24585 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:44.248381 24585 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:33:44.248404 24585 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:44.256982 24585 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.66:37575
I20260929 10:33:44.257012 24713 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.66:37575 every 8 connection(s)
I20260929 10:33:44.258045 24585 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
I20260929 10:33:44.263482 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 24585
I20260929 10:33:44.263933 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.67:43203
--local_ip_for_outbound_sockets=127.15.82.67
--tserver_master_addrs=127.15.82.126:33713
--webserver_port=40465
--webserver_interface=127.15.82.67
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab
--principal=kudu/127.15.82.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:46833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.15.82.67
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/krb5kdc/oryx_127.15.82.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 10:33:44 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.66@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:33:44 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.66@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
W20260929 10:33:44.269879 24583 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:33:44.259837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:39323 (local address 127.15.82.126:33713)
0929 10:33:44.259983 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:44.259987 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:44.260837 (+ 850us) server_negotiation.cc:407] Connection header received
0929 10:33:44.262142 (+ 1305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:44.262145 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:44.262206 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:44.262316 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:44.263517 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:44.264178 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:44.265398 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:44.265623 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:44.265720 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:44.269775 (+ 4055us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.82.66:39323: BlockingRecv error: recv got EOF from 127.15.82.66:39323 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
I20260929 10:33:44.270848 24716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:44.260238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:33713 (local address 127.15.82.66:39323)
0929 10:33:44.260688 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:44.260731 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:44.261928 (+ 1197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:44.262527 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:44.262534 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:44.262894 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:44.263365 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:44.263379 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:44.264305 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:44.264314 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:44.265232 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:44.265252 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:44.265791 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:44.265797 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:44.266025 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:44.266720 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:44.266746 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:44.269663 (+ 2917us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.82.126:33713: Server oryx/127.15.82.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":115,"threads_started":1}
W20260929 10:33:44.271591 24714 heartbeater.cc:644] Failed to heartbeat to 127.15.82.126:33713 (0 consecutive failures): Not authorized: Failed to ping master at 127.15.82.126:33713: Client connection negotiation failed: client connection to 127.15.82.126:33713: Server oryx/127.15.82.126@KRBTEST.COM not found in Kerberos database
Sep 29 10:33:44 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.66@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:33:44 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.66@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:33:44.277571 24716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:44.272024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:33713 (local address 127.15.82.66:40431)
0929 10:33:44.272235 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:44.272247 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:44.272310 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:44.272551 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:44.272553 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:44.272681 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:44.272773 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:44.272777 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:44.273875 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:44.273877 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:44.274502 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:44.274511 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:44.275138 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:44.275141 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:44.275205 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:44.275735 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:44.275745 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:44.277470 (+ 1725us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.82.126:33713: Server oryx/127.15.82.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135}
W20260929 10:33:44.277750 24583 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:33:44.272145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:40431 (local address 127.15.82.126:33713)
0929 10:33:44.272298 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:44.272304 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:44.272323 (+ 19us) server_negotiation.cc:407] Connection header received
0929 10:33:44.272376 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:44.272382 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:44.272436 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:44.272545 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:44.272882 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:44.273768 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:44.274613 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:44.275028 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:44.275159 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:44.277636 (+ 2477us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.82.66:40431: BlockingRecv error: recv got EOF from 127.15.82.66:40431 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Sep 29 10:33:44 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.66@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:33:44 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.66@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:33:44.283314 24716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:44.278100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:33713 (local address 127.15.82.66:33351)
0929 10:33:44.278221 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:44.278236 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:44.278312 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:44.278594 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:44.278597 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:44.278754 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:44.278841 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:44.278847 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:44.279510 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:44.279513 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:44.280384 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:44.280393 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:44.280785 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:44.280789 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:44.280870 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:44.281314 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:44.281327 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:44.283171 (+ 1844us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.82.126:33713: Server oryx/127.15.82.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":45}
W20260929 10:33:44.283514 24583 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:33:44.278174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:33351 (local address 127.15.82.126:33713)
0929 10:33:44.278279 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:44.278282 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:44.278294 (+ 12us) server_negotiation.cc:407] Connection header received
0929 10:33:44.278442 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:44.278445 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:44.278490 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:44.278556 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:44.278943 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:44.279415 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:44.280494 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:44.280688 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:44.280759 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:44.283283 (+ 2524us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.82.66:33351: BlockingRecv error: recv got EOF from 127.15.82.66:33351 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260929 10:33:44.283516 24714 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 10:33:44.379711 24717 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:33:44.380034 24717 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:33:44.380117 24717 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:33:44.384163 24717 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:33:44.384264 24717 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:33:44.384361 24717 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.67
I20260929 10:33:44.389631 24717 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:46833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/krb5kdc/oryx_127.15.82.67.keytab
--principal=oryx/127.15.82.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.67:43203
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.82.67
--webserver_port=40465
--tserver_master_addrs=127.15.82.126:33713
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24717
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.67
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:33:44.390803 24717 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:33:44.391918 24717 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:33:44.402758 24722 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:44.402777 24723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:44.402758 24725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:44.403066 24717 server_base.cc:1034] running on GCE node
I20260929 10:33:44.403470 24717 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:33:44.404122 24717 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:33:44.405267 24717 hybrid_clock.cc:648] HybridClock initialized: now 1790678024405243 us; error 38 us; skew 500 ppm
Sep 29 10:33:44 dist-test-slave-vg89 krb5kdc[23850](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678024, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.82.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:33:44.408233 24717 init.cc:377] Logged in from keytab as oryx/127.15.82.67@KRBTEST.COM (short username oryx)
I20260929 10:33:44.409162 24717 webserver.cc:460] Webserver started at http://127.15.82.67:40465/ using document root <none> and password file <none>
I20260929 10:33:44.409817 24717 fs_manager.cc:359] Metadata directory not provided
I20260929 10:33:44.409893 24717 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:33:44.413721 24717 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260929 10:33:44.416090 24732 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:44.417361 24717 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 10:33:44.417481 24717 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-2/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-2/wal
uuid: "9be40c3c5ad1422e8221e7f285cb4d1e"
format_stamp: "Formatted at 2026-09-29 10:33:43 on dist-test-slave-vg89"
server_key: "043cf0e67b0ccddd4d745b7269511f15"
server_key_iv: "0ed236f73115e205f480059796164c7e"
server_key_version: "encryptionkey@0"
I20260929 10:33:44.418046 24717 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:33:44.434981 24717 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:33:44.438459 24717 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:33:44.438668 24717 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:33:44.439291 24717 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:33:44.440176 24717 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:33:44.440222 24717 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:44.440289 24717 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:33:44.440323 24717 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:44.449631 24717 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.67:43203
I20260929 10:33:44.449654 24845 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.67:43203 every 8 connection(s)
I20260929 10:33:44.450743 24717 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
I20260929 10:33:44.451575 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 24717
Sep 29 10:33:44 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.67@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:33:44.462023 24583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:44.452324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36648 (local address 127.15.82.126:33713)
0929 10:33:44.452495 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:44.452500 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:44.452752 (+ 252us) server_negotiation.cc:407] Connection header received
0929 10:33:44.452909 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:44.452912 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:44.452967 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:44.453070 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:44.453907 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:44.454606 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:44.455804 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:44.456100 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:44.456178 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:44.457854 (+ 1676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:44.457860 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:44.457878 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:33:44.457908 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:44.459861 (+ 1953us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:44.460345 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:44.460349 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:44.460354 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:44.460471 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:33:44.460788 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:33:44.460793 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:33:44.460795 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:33:44.461157 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:44.461371 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:44.461649 (+ 278us) server_negotiation.cc:299] Negotiation successful
0929 10:33:44.461897 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
Sep 29 10:33:44 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.67@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
W20260929 10:33:44.463853 24849 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:33:44.453085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.67:44835 (local address 127.15.82.126:33713)
0929 10:33:44.453311 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:44.453315 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:44.454033 (+ 718us) server_negotiation.cc:407] Connection header received
0929 10:33:44.455703 (+ 1670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:44.455706 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:44.455786 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:44.455878 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:44.457468 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:44.458105 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:44.459669 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:44.460056 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:44.460130 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:44.463731 (+ 3601us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.82.67:44835: BlockingRecv error: recv got EOF from 127.15.82.67:44835 (error 108)
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":64,"threads_started":1}
I20260929 10:33:44.464633 24850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:44.453345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:33713 (local address 127.15.82.67:44835)
0929 10:33:44.453879 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:44.453933 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:44.455447 (+ 1514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:44.456058 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:44.456065 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:44.456591 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:44.457298 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:44.457322 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:44.458257 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:44.458263 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:44.459528 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:44.459537 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:44.460205 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:44.460209 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:44.460422 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:44.461030 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:44.461046 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:44.463629 (+ 2583us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.82.126:33713: Server oryx/127.15.82.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":121,"threads_started":1}
W20260929 10:33:44.465088 24846 heartbeater.cc:644] Failed to heartbeat to 127.15.82.126:33713 (0 consecutive failures): Not authorized: Failed to ping master at 127.15.82.126:33713: Client connection negotiation failed: client connection to 127.15.82.126:33713: Server oryx/127.15.82.126@KRBTEST.COM not found in Kerberos database
Sep 29 10:33:44 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.67@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:33:44 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.67@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:33:44.470230 24850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:44.465528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:33713 (local address 127.15.82.67:49241)
0929 10:33:44.465646 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:44.465660 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:44.465733 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:44.465918 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:44.465920 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:44.466055 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:44.466151 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:44.466155 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:44.466878 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:44.466881 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:44.467429 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:44.467437 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:44.467821 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:44.467823 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:44.467890 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:44.468390 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:44.468402 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:44.470133 (+ 1731us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.82.126:33713: Server oryx/127.15.82.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":45}
W20260929 10:33:44.470376 24849 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:33:44.465590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.67:49241 (local address 127.15.82.126:33713)
0929 10:33:44.465717 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:44.465720 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:44.465731 (+ 11us) server_negotiation.cc:407] Connection header received
0929 10:33:44.465792 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:44.465793 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:44.465826 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:44.465888 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:44.466244 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:44.466789 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:44.467529 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:44.467731 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:44.467802 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:44.470261 (+ 2459us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.82.67:49241: BlockingRecv error: recv got EOF from 127.15.82.67:49241 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Sep 29 10:33:44 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.67@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:33:44 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.67@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:33:44.475680 24850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:44.470893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:33713 (local address 127.15.82.67:41441)
0929 10:33:44.471004 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:44.471019 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:44.471093 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:44.471419 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:44.471421 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:44.471562 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:44.471651 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:44.471656 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:44.472312 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:44.472314 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:44.472851 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:44.472858 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:44.473286 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:44.473289 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:44.473344 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:44.473754 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:44.473765 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:44.475553 (+ 1788us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.82.126:33713: Server oryx/127.15.82.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":47}
W20260929 10:33:44.475816 24849 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:33:44.471099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.67:41441 (local address 127.15.82.126:33713)
0929 10:33:44.471241 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:44.471244 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:44.471255 (+ 11us) server_negotiation.cc:407] Connection header received
0929 10:33:44.471287 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:44.471289 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:44.471322 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:44.471387 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:44.471745 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:44.472223 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:44.472950 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:44.473194 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:44.473271 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:44.475700 (+ 2429us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.82.67:41441: BlockingRecv error: recv got EOF from 127.15.82.67:41441 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260929 10:33:44.476171 24846 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 29 10:33:45 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.65@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:33:45 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.65@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:33:45.104254 24851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:45.099153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:33713 (local address 127.15.82.65:35915)
0929 10:33:45.099435 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:45.099449 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:45.099549 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:45.099808 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:45.099811 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:45.099990 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:45.100107 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:45.100112 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:45.100814 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:45.100816 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:45.101368 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:45.101374 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:45.101768 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:45.101770 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:45.101840 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:45.102307 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:45.102318 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:45.104141 (+ 1823us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.82.126:33713: Server oryx/127.15.82.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":88,"threads_started":1}
W20260929 10:33:45.104450 24852 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:33:45.099234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.65:35915 (local address 127.15.82.126:33713)
0929 10:33:45.099479 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:45.099484 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:45.099497 (+ 13us) server_negotiation.cc:407] Connection header received
0929 10:33:45.099651 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:45.099653 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:45.099697 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:45.099773 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:45.100208 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:45.100725 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:45.101463 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:45.101675 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:45.101747 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:45.104271 (+ 2524us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.82.65:35915: BlockingRecv error: recv got EOF from 127.15.82.65:35915 (error 108)
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":76,"threads_started":1}
Sep 29 10:33:45 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.66@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:33:45 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.66@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:33:45.290117 24853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:45.284193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:33713 (local address 127.15.82.66:58649)
0929 10:33:45.284535 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:45.284584 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:45.284713 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:45.285030 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:45.285034 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:45.285302 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:45.285455 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:45.285464 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:45.286342 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:45.286345 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:45.286946 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:45.286955 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:45.287371 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:45.287374 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:45.287438 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:45.288019 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:45.288033 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:45.289979 (+ 1946us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.82.126:33713: Server oryx/127.15.82.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":84,"threads_started":1}
W20260929 10:33:45.290401 24852 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:33:45.284337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:58649 (local address 127.15.82.126:33713)
0929 10:33:45.284522 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:45.284530 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:45.284671 (+ 141us) server_negotiation.cc:407] Connection header received
0929 10:33:45.284820 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:45.284823 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:45.284884 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:45.284994 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:45.285601 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:45.286229 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:45.287052 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:45.287268 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:45.287348 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:45.290148 (+ 2800us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.82.66:58649: BlockingRecv error: recv got EOF from 127.15.82.66:58649 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Sep 29 10:33:45 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.67@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:33:45 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.67@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:33:45.482371 24854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:45.476777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:33713 (local address 127.15.82.67:42605)
0929 10:33:45.477096 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:45.477111 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:45.477252 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:45.477544 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:45.477547 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:45.477757 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:45.477871 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:45.477876 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:45.478701 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:45.478705 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:45.479294 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:45.479301 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:45.479775 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:45.479779 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:45.479878 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:45.480374 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:45.480386 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:45.482200 (+ 1814us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.82.126:33713: Server oryx/127.15.82.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":84,"threads_started":1}
W20260929 10:33:45.482465 24852 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:33:45.476888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.67:42605 (local address 127.15.82.126:33713)
0929 10:33:45.477059 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:45.477064 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:45.477199 (+ 135us) server_negotiation.cc:407] Connection header received
0929 10:33:45.477362 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:45.477366 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:45.477421 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:45.477509 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:45.477996 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:45.478592 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:45.479417 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:45.479662 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:45.479773 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:45.482324 (+ 2551us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.82.67:42605: BlockingRecv error: recv got EOF from 127.15.82.67:42605 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Sep 29 10:33:46 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.65@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:33:46 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.65@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:33:46.110474 24855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:46.105115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:33713 (local address 127.15.82.65:58429)
0929 10:33:46.105397 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:46.105411 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:46.105526 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:46.105764 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:46.105766 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:46.105948 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:46.106049 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:46.106054 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:46.106783 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:46.106786 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:46.107329 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:46.107336 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:46.107754 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:46.107756 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:46.107816 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:46.108313 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:46.108324 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:46.110271 (+ 1947us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.82.126:33713: Server oryx/127.15.82.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":89,"threads_started":1}
W20260929 10:33:46.110544 24856 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:33:46.105249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.65:58429 (local address 127.15.82.126:33713)
0929 10:33:46.105546 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:46.105550 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:46.105563 (+ 13us) server_negotiation.cc:407] Connection header received
0929 10:33:46.105599 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:46.105601 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:46.105642 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:46.105724 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:46.106150 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:46.106690 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:46.107433 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:46.107656 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:46.107735 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:46.110394 (+ 2659us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.82.65:58429: BlockingRecv error: recv got EOF from 127.15.82.65:58429 (error 108)
Metrics: {"server-negotiator.queue_time_us":206,"thread_start_us":103,"threads_started":1}
Sep 29 10:33:46 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.66@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:33:46 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.66@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:33:46.301563 24857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:46.291003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:33713 (local address 127.15.82.66:57415)
0929 10:33:46.291302 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:46.291319 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:46.291452 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:46.291737 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:46.291740 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:46.291946 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:46.292058 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:46.292063 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:46.292806 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:46.292811 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:46.293384 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:46.293391 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:46.293907 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:46.293910 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:46.294000 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:46.299626 (+ 5626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:46.299638 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:46.301429 (+ 1791us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.82.126:33713: Server oryx/127.15.82.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":87,"threads_started":1}
W20260929 10:33:46.301719 24856 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:33:46.291092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:57415 (local address 127.15.82.126:33713)
0929 10:33:46.291262 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:46.291266 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:46.291399 (+ 133us) server_negotiation.cc:407] Connection header received
0929 10:33:46.291577 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:46.291580 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:46.291632 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:46.291717 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:46.292172 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:46.292698 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:46.293521 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:46.293771 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:46.293896 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:46.301565 (+ 7669us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.82.66:57415: BlockingRecv error: recv got EOF from 127.15.82.66:57415 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Sep 29 10:33:46 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.67@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:33:46 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.67@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:33:46.488168 24858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:46.483228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:33713 (local address 127.15.82.67:39419)
0929 10:33:46.483500 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:46.483513 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:46.483607 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:46.483825 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:46.483827 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:46.484001 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:46.484095 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:46.484100 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:46.484779 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:46.484781 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:46.485317 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:46.485324 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:46.485709 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:46.485711 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:46.485773 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:46.486256 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:46.486266 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:46.488052 (+ 1786us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.82.126:33713: Server oryx/127.15.82.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":94,"threads_started":1}
W20260929 10:33:46.488332 24856 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:33:46.483370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.67:39419 (local address 127.15.82.126:33713)
0929 10:33:46.483506 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:46.483509 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:46.483563 (+ 54us) server_negotiation.cc:407] Connection header received
0929 10:33:46.483682 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:46.483684 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:46.483726 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:46.483786 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:46.484191 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:46.484689 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:46.485411 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:46.485616 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:46.485690 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:46.488176 (+ 2486us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.82.67:39419: BlockingRecv error: recv got EOF from 127.15.82.67:39419 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Sep 29 10:33:47 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.65@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:33:47 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.65@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:33:47.123822 24859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:47.111270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:33713 (local address 127.15.82.65:44313)
0929 10:33:47.111576 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:47.111590 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:47.111686 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:47.111936 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:47.111938 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:47.112115 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:47.112212 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:47.112217 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:47.112906 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:47.112909 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:47.113463 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:47.113470 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:47.114073 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:47.114077 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:47.114172 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:47.121720 (+ 7548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:47.121734 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:47.123690 (+ 1956us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.82.126:33713: Server oryx/127.15.82.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":76,"threads_started":1}
W20260929 10:33:47.124040 24860 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:33:47.111334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.65:44313 (local address 127.15.82.126:33713)
0929 10:33:47.111614 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:47.111618 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:47.111635 (+ 17us) server_negotiation.cc:407] Connection header received
0929 10:33:47.111768 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:47.111770 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:47.111812 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:47.111887 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:47.112304 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:47.112817 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:47.113631 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:47.113943 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:47.114065 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:47.123851 (+ 9786us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.82.65:44313: BlockingRecv error: recv got EOF from 127.15.82.65:44313 (error 108)
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":108,"threads_started":1}
Sep 29 10:33:47 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.66@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:33:47 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.66@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:33:47.307670 24861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:47.302462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:33713 (local address 127.15.82.66:49651)
0929 10:33:47.302758 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:47.302771 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:47.302872 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:47.303123 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:47.303125 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:47.303298 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:47.303392 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:47.303397 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:47.304107 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:47.304110 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:47.304663 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:47.304670 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:47.305084 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:47.305087 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:47.305171 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:47.305647 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:47.305658 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:47.307546 (+ 1888us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.82.126:33713: Server oryx/127.15.82.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":87,"threads_started":1}
W20260929 10:33:47.307955 24860 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:33:47.302579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:49651 (local address 127.15.82.126:33713)
0929 10:33:47.302764 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:47.302768 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:47.302844 (+ 76us) server_negotiation.cc:407] Connection header received
0929 10:33:47.302954 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:47.302961 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:47.303019 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:47.303088 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:47.303491 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:47.304012 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:47.304761 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:47.304967 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:47.305051 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:47.307715 (+ 2664us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.82.66:49651: BlockingRecv error: recv got EOF from 127.15.82.66:49651 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
Sep 29 10:33:47 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.67@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:33:47 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.67@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:33:47.494081 24862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:47.488895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:33713 (local address 127.15.82.67:33451)
0929 10:33:47.489160 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:47.489173 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:47.489265 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:47.489499 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:47.489501 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:47.489672 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:47.489768 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:47.489773 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:47.490491 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:47.490494 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:47.491049 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:47.491056 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:47.491458 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:47.491461 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:47.491523 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:47.492005 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:47.492015 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:47.493886 (+ 1871us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.82.126:33713: Server oryx/127.15.82.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":95,"threads_started":1}
W20260929 10:33:47.494370 24860 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:33:47.488952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.67:33451 (local address 127.15.82.126:33713)
0929 10:33:47.489102 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:47.489105 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:47.489240 (+ 135us) server_negotiation.cc:407] Connection header received
0929 10:33:47.489343 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:47.489345 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:47.489389 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:47.489457 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:47.489863 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:47.490414 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:47.491146 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:47.491357 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:47.491430 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:47.494072 (+ 2642us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.82.67:33451: BlockingRecv error: recv got EOF from 127.15.82.67:33451 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Sep 29 10:33:48 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.65@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:33:48 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.65@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:33:48.129838 24863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:48.124539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:33713 (local address 127.15.82.65:36957)
0929 10:33:48.124805 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:48.124818 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:48.124907 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:48.125154 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:48.125157 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:48.125332 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:48.125427 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:48.125432 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:48.126214 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:48.126217 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:48.126791 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:48.126798 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:48.127297 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:48.127300 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:48.127364 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:48.127931 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:48.127943 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:48.129723 (+ 1780us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.82.126:33713: Server oryx/127.15.82.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":98,"threads_started":1}
W20260929 10:33:48.129987 24864 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:33:48.124607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.65:36957 (local address 127.15.82.126:33713)
0929 10:33:48.124848 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:48.124852 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:48.124864 (+ 12us) server_negotiation.cc:407] Connection header received
0929 10:33:48.124994 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:48.124996 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:48.125038 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:48.125118 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:48.125536 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:48.126121 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:48.126884 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:48.127197 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:48.127276 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:48.129846 (+ 2570us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.82.65:36957: BlockingRecv error: recv got EOF from 127.15.82.65:36957 (error 108)
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":94,"threads_started":1}
Sep 29 10:33:48 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.66@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:33:48 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.66@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:33:48.313660 24865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:48.308561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:33713 (local address 127.15.82.66:58617)
0929 10:33:48.308830 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:48.308842 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:48.308945 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:48.309178 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:48.309180 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:48.309354 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:48.309456 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:48.309461 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:48.310141 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:48.310144 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:48.310873 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:48.310883 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:48.311298 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:48.311300 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:48.311360 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:48.311814 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:48.311825 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:48.313529 (+ 1704us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.82.126:33713: Server oryx/127.15.82.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":83,"threads_started":1}
W20260929 10:33:48.313818 24864 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:33:48.308655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:58617 (local address 127.15.82.126:33713)
0929 10:33:48.308841 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:48.308845 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:48.308913 (+ 68us) server_negotiation.cc:407] Connection header received
0929 10:33:48.309028 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:48.309030 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:48.309073 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:48.309143 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:48.309551 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:48.310054 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:48.310990 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:48.311204 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:48.311281 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:48.313662 (+ 2381us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.82.66:58617: BlockingRecv error: recv got EOF from 127.15.82.66:58617 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
Sep 29 10:33:48 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.67@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:33:48 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.67@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:33:48.500591 24866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:48.495031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:33713 (local address 127.15.82.67:57573)
0929 10:33:48.495322 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:48.495336 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:48.495482 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:48.495762 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:48.495765 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:48.495977 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:48.496072 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:48.496078 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:48.496837 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:48.496842 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:48.497396 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:48.497403 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:48.497886 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:48.497889 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:48.497981 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:48.498613 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:48.498624 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:48.500399 (+ 1775us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.82.126:33713: Server oryx/127.15.82.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":87,"threads_started":1}
W20260929 10:33:48.500638 24864 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:33:48.495097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.67:57573 (local address 127.15.82.126:33713)
0929 10:33:48.495264 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:48.495268 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:48.495410 (+ 142us) server_negotiation.cc:407] Connection header received
0929 10:33:48.495587 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:48.495590 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:48.495647 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:48.495735 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:48.496190 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:48.496726 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:48.497526 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:48.497764 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:48.497872 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:48.500530 (+ 2658us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.82.67:57573: BlockingRecv error: recv got EOF from 127.15.82.67:57573 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Sep 29 10:33:49 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.65@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:33:49 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.65@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:33:49.136202 24867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:49.130714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:33713 (local address 127.15.82.65:47133)
0929 10:33:49.131008 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:49.131022 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:49.131110 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:49.131391 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:49.131394 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:49.131619 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:49.131717 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:49.131724 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:49.132523 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:49.132526 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:49.133145 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:49.133154 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:49.133555 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:49.133558 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:49.133621 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:49.134184 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:49.134197 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:49.136031 (+ 1834us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.82.126:33713: Server oryx/127.15.82.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":69,"threads_started":1}
W20260929 10:33:49.136273 24868 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:33:49.130824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.65:47133 (local address 127.15.82.126:33713)
0929 10:33:49.131059 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:49.131063 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:49.131075 (+ 12us) server_negotiation.cc:407] Connection header received
0929 10:33:49.131193 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:49.131195 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:49.131237 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:49.131312 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:49.131813 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:49.132412 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:49.133242 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:49.133459 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:49.133534 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:49.136151 (+ 2617us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.82.65:47133: BlockingRecv error: recv got EOF from 127.15.82.65:47133 (error 108)
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":92,"threads_started":1}
Sep 29 10:33:49 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.66@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:33:49 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.66@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:33:49.320012 24869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:49.314469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:33713 (local address 127.15.82.66:41461)
0929 10:33:49.314753 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:49.314768 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:49.314880 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:49.315165 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:49.315168 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:49.315376 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:49.315466 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:49.315471 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:49.316229 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:49.316234 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:49.316808 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:49.316815 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:49.317313 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:49.317317 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:49.317413 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:49.317870 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:49.317881 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:49.319882 (+ 2001us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.82.126:33713: Server oryx/127.15.82.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":76,"threads_started":1}
W20260929 10:33:49.320272 24868 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:33:49.314624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:41461 (local address 127.15.82.126:33713)
0929 10:33:49.314768 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:49.314773 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:49.314822 (+ 49us) server_negotiation.cc:407] Connection header received
0929 10:33:49.314995 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:49.314999 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:49.315047 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:49.315139 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:49.315578 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:49.316115 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:49.316931 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:49.317173 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:49.317278 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:49.320020 (+ 2742us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.82.66:41461: BlockingRecv error: recv got EOF from 127.15.82.66:41461 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Sep 29 10:33:49 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.67@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:33:49 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.67@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:33:49.506959 24870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:49.501606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:33713 (local address 127.15.82.67:47809)
0929 10:33:49.502023 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:49.502035 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:49.502176 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:49.502427 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:49.502429 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:49.502648 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:49.502744 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:49.502749 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:49.503451 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:49.503455 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:49.504025 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:49.504032 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:49.504425 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:49.504427 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:49.504492 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:49.504968 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:49.504979 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:49.506839 (+ 1860us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.82.126:33713: Server oryx/127.15.82.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":88,"threads_started":1}
W20260929 10:33:49.507105 24868 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:33:49.501668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.67:47809 (local address 127.15.82.126:33713)
0929 10:33:49.501819 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:49.501825 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:49.502129 (+ 304us) server_negotiation.cc:407] Connection header received
0929 10:33:49.502261 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:49.502263 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:49.502309 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:49.502373 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:49.502844 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:49.503362 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:49.504123 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:49.504334 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:49.504407 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:49.506986 (+ 2579us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.82.67:47809: BlockingRecv error: recv got EOF from 127.15.82.67:47809 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Sep 29 10:33:50 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.65@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:33:50 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.65@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:33:50.146309 24871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:50.137061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:33713 (local address 127.15.82.65:40105)
0929 10:33:50.140495 (+ 3434us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:50.140513 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:50.140648 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:50.140948 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:50.140951 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:50.141177 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:50.141295 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:50.141304 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:50.142551 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:50.142555 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:50.143254 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:50.143264 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:50.143693 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:50.143695 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:50.143756 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:50.144289 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:50.144299 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:50.146193 (+ 1894us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.82.126:33713: Server oryx/127.15.82.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":3321,"thread_start_us":98,"threads_started":1}
W20260929 10:33:50.146492 24872 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:33:50.137472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.65:40105 (local address 127.15.82.126:33713)
0929 10:33:50.137702 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:50.137707 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:50.140613 (+ 2906us) server_negotiation.cc:407] Connection header received
0929 10:33:50.140746 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:50.140748 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:50.140805 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:50.140928 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:50.141625 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:50.142452 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:50.143356 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:50.143585 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:50.143672 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:50.146325 (+ 2653us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.82.65:40105: BlockingRecv error: recv got EOF from 127.15.82.65:40105 (error 108)
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":68,"threads_started":1}
Sep 29 10:33:50 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.66@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:33:50 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.66@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:33:50.326635 24873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:50.320781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:33713 (local address 127.15.82.66:46963)
0929 10:33:50.321190 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:50.321205 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:50.321349 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:50.321625 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:50.321628 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:50.321881 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:50.321970 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:50.321975 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:50.322884 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:50.322889 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:50.323460 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:50.323467 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:50.323934 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:50.323938 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:50.324026 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:50.324498 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:50.324509 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:50.326450 (+ 1941us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.82.126:33713: Server oryx/127.15.82.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":93,"threads_started":1}
W20260929 10:33:50.326692 24872 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:33:50.320909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:46963 (local address 127.15.82.126:33713)
0929 10:33:50.321132 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:50.321136 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:50.321294 (+ 158us) server_negotiation.cc:407] Connection header received
0929 10:33:50.321458 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:50.321461 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:50.321516 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:50.321605 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:50.322098 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:50.322769 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:50.323581 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:50.323821 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:50.323932 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:50.326574 (+ 2642us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.82.66:46963: BlockingRecv error: recv got EOF from 127.15.82.66:46963 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Sep 29 10:33:50 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.67@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:33:50 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.67@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:33:50.513378 24874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:50.508022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:33713 (local address 127.15.82.67:54891)
0929 10:33:50.508315 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:50.508329 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:50.508429 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:50.508646 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:50.508648 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:50.508830 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:50.508925 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:50.508929 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:50.509650 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:50.509653 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:50.510204 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:50.510211 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:50.510632 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:50.510635 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:50.510703 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:50.511230 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:50.511241 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:50.513186 (+ 1945us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.82.126:33713: Server oryx/127.15.82.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":102,"threads_started":1}
W20260929 10:33:50.513512 24872 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:33:50.508108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.67:54891 (local address 127.15.82.126:33713)
0929 10:33:50.508295 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:50.508299 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:50.508419 (+ 120us) server_negotiation.cc:407] Connection header received
0929 10:33:50.508513 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:50.508515 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:50.508562 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:50.508635 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:50.509029 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:50.509554 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:50.510305 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:50.510549 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:50.510624 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:50.513376 (+ 2752us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.82.67:54891: BlockingRecv error: recv got EOF from 127.15.82.67:54891 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Sep 29 10:33:51 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.65@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:33:51 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.65@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:33:51.152336 24875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:51.147097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:33713 (local address 127.15.82.65:51837)
0929 10:33:51.147426 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:51.147451 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:51.147556 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:51.147800 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:51.147802 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:51.147992 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:51.148083 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:51.148088 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:51.148810 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:51.148812 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:51.149350 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:51.149357 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:51.149752 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:51.149754 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:51.149815 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:51.150303 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:51.150314 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:51.152217 (+ 1903us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.82.126:33713: Server oryx/127.15.82.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":83,"threads_started":1}
W20260929 10:33:51.152522 24876 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:33:51.147165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.65:51837 (local address 127.15.82.126:33713)
0929 10:33:51.147429 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:51.147433 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:51.147525 (+ 92us) server_negotiation.cc:407] Connection header received
0929 10:33:51.147638 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:51.147640 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:51.147684 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:51.147760 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:51.148188 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:51.148719 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:51.149443 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:51.149660 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:51.149732 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:51.152352 (+ 2620us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.82.65:51837: BlockingRecv error: recv got EOF from 127.15.82.65:51837 (error 108)
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":70,"threads_started":1}
Sep 29 10:33:51 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.66@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:33:51 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.66@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:33:51.333459 24877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:51.327640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:33713 (local address 127.15.82.66:38657)
0929 10:33:51.327994 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:51.328007 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:51.328141 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:51.328419 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:51.328422 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:51.328661 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:51.328754 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:51.328760 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:51.329534 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:51.329539 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:51.330167 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:51.330176 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:51.330717 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:51.330721 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:51.330809 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:51.331381 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:51.331392 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:51.333334 (+ 1942us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.82.126:33713: Server oryx/127.15.82.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":98,"threads_started":1}
W20260929 10:33:51.333640 24876 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:33:51.327708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:38657 (local address 127.15.82.126:33713)
0929 10:33:51.327894 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:51.327899 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:51.328092 (+ 193us) server_negotiation.cc:407] Connection header received
0929 10:33:51.328251 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:51.328254 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:51.328310 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:51.328399 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:51.328878 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:51.329421 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:51.330264 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:51.330599 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:51.330714 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:51.333480 (+ 2766us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.82.66:38657: BlockingRecv error: recv got EOF from 127.15.82.66:38657 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
Sep 29 10:33:51 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.67@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:33:51 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.67@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:33:51.520534 24878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:51.514508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:33713 (local address 127.15.82.67:44325)
0929 10:33:51.514866 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:51.514882 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:51.514994 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:51.515269 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:51.515272 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:51.515472 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:51.515576 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:51.515582 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:51.516354 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:51.516357 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:51.516955 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:51.516964 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:51.517465 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:51.517468 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:51.517537 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:51.518077 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:51.518089 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:51.520345 (+ 2256us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.82.126:33713: Server oryx/127.15.82.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":119,"threads_started":1}
W20260929 10:33:51.520597 24876 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:33:51.514550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.67:44325 (local address 127.15.82.126:33713)
0929 10:33:51.514682 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:51.514685 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:51.514957 (+ 272us) server_negotiation.cc:407] Connection header received
0929 10:33:51.515094 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:51.515096 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:51.515143 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:51.515216 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:51.515683 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:51.516213 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:51.517082 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:51.517310 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:51.517394 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:51.520483 (+ 3089us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.82.67:44325: BlockingRecv error: recv got EOF from 127.15.82.67:44325 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Sep 29 10:33:52 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.65@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:33:52 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.65@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:33:52.168823 24879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:52.153145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:33713 (local address 127.15.82.65:54309)
0929 10:33:52.153426 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:52.153447 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:52.153536 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:52.153788 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:52.153790 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:52.153960 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:52.154058 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:52.154063 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:52.154845 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:52.154847 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:52.155411 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:52.155418 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:52.155865 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:52.155868 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:52.155927 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:52.166821 (+ 10894us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:52.166834 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:52.168703 (+ 1869us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.82.126:33713: Server oryx/127.15.82.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":65,"threads_started":1}
W20260929 10:33:52.168987 24880 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:33:52.153288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.65:54309 (local address 127.15.82.126:33713)
0929 10:33:52.153528 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:52.153531 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:52.153544 (+ 13us) server_negotiation.cc:407] Connection header received
0929 10:33:52.153629 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:52.153631 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:52.153674 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:52.153750 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:52.154157 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:52.154751 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:52.155504 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:52.155768 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:52.155847 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:52.168847 (+ 13000us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.82.65:54309: BlockingRecv error: recv got EOF from 127.15.82.65:54309 (error 108)
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":71,"threads_started":1}
Sep 29 10:33:52 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.66@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:33:52 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.66@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:33:52.339944 24881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:52.334223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:33713 (local address 127.15.82.66:34831)
0929 10:33:52.334581 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:52.334595 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:52.334751 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:52.335038 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:52.335041 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:52.335243 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:52.335336 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:52.335341 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:52.336289 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:52.336294 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:52.336876 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:52.336883 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:52.337398 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:52.337402 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:52.337482 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:52.337991 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:52.338002 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:52.339822 (+ 1820us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.82.126:33713: Server oryx/127.15.82.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":85,"threads_started":1}
W20260929 10:33:52.340121 24880 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:33:52.334348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:34831 (local address 127.15.82.126:33713)
0929 10:33:52.334532 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:52.334537 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:52.334684 (+ 147us) server_negotiation.cc:407] Connection header received
0929 10:33:52.334859 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:52.334862 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:52.334917 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:52.335011 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:52.335467 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:52.336152 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:52.337006 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:52.337268 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:52.337405 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:52.339983 (+ 2578us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.82.66:34831: BlockingRecv error: recv got EOF from 127.15.82.66:34831 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Sep 29 10:33:52 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.67@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:33:52 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.67@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:33:52.527205 24882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:52.521539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:33713 (local address 127.15.82.67:55355)
0929 10:33:52.521809 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:52.521826 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:52.521983 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:52.522254 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:52.522257 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:52.522494 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:52.522587 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:52.522593 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:52.523351 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:52.523357 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:52.523922 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:52.523930 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:52.524479 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:52.524483 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:52.524569 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:52.525110 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:52.525122 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:52.527067 (+ 1945us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.82.126:33713: Server oryx/127.15.82.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":79,"threads_started":1}
W20260929 10:33:52.527364 24880 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:33:52.521604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.67:55355 (local address 127.15.82.126:33713)
0929 10:33:52.521790 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:52.521794 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:52.521920 (+ 126us) server_negotiation.cc:407] Connection header received
0929 10:33:52.522086 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:52.522089 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:52.522143 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:52.522232 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:52.522703 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:52.523243 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:52.524036 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:52.524349 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:52.524477 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:52.527203 (+ 2726us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.82.67:55355: BlockingRecv error: recv got EOF from 127.15.82.67:55355 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Sep 29 10:33:53 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.65@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:33:53 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.65@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:33:53.175968 24883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:53.169713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:33713 (local address 127.15.82.65:51119)
0929 10:33:53.170051 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:53.170066 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:53.170173 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:53.170561 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:53.170567 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:53.170829 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:53.170948 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:53.170955 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:53.171854 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:53.171857 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:53.172435 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:53.172443 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:53.172905 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:53.172909 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:53.172981 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:53.173585 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:53.173601 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:53.175825 (+ 2224us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.82.126:33713: Server oryx/127.15.82.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":130,"threads_started":1}
W20260929 10:33:53.176134 24884 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:33:53.169787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.65:51119 (local address 127.15.82.126:33713)
0929 10:33:53.170072 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:53.170076 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:53.170127 (+ 51us) server_negotiation.cc:407] Connection header received
0929 10:33:53.170296 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:53.170303 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:53.170350 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:53.170495 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:53.171129 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:53.171747 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:53.172542 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:53.172796 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:53.172893 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:53.175983 (+ 3090us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.82.65:51119: BlockingRecv error: recv got EOF from 127.15.82.65:51119 (error 108)
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":95,"threads_started":1}
Sep 29 10:33:53 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.66@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:33:53 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.66@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:33:53.346084 24885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:53.340753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:33713 (local address 127.15.82.66:47715)
0929 10:33:53.341033 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:53.341046 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:53.341144 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:53.341368 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:53.341370 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:53.341561 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:53.341656 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:53.341661 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:53.342371 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:53.342373 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:53.342956 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:53.342965 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:53.343388 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:53.343391 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:53.343463 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:53.343953 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:53.343965 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:53.345952 (+ 1987us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.82.126:33713: Server oryx/127.15.82.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":92,"threads_started":1}
W20260929 10:33:53.346257 24884 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:33:53.340820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:47715 (local address 127.15.82.126:33713)
0929 10:33:53.340981 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:53.340991 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:53.341116 (+ 125us) server_negotiation.cc:407] Connection header received
0929 10:33:53.341222 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:53.341224 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:53.341267 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:53.341332 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:53.341753 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:53.342282 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:53.343056 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:53.343272 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:53.343349 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:53.346098 (+ 2749us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.82.66:47715: BlockingRecv error: recv got EOF from 127.15.82.66:47715 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Sep 29 10:33:53 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.67@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:33:53 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.67@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:33:53.533561 24886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:53.528100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:33713 (local address 127.15.82.67:46443)
0929 10:33:53.528444 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:53.528457 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:53.528553 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:53.528793 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:53.528795 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:53.528977 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:53.529073 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:53.529079 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:53.529899 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:53.529902 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:53.530466 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:53.530473 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:53.530883 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:53.530887 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:53.530949 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:53.531458 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:53.531468 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:53.533438 (+ 1970us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.82.126:33713: Server oryx/127.15.82.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":101,"threads_started":1}
W20260929 10:33:53.533821 24884 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:33:53.528187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.67:46443 (local address 127.15.82.126:33713)
0929 10:33:53.528375 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:53.528379 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:53.528523 (+ 144us) server_negotiation.cc:407] Connection header received
0929 10:33:53.528635 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:53.528637 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:53.528685 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:53.528745 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:53.529170 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:53.529792 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:53.530582 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:53.530790 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:53.530864 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:53.533579 (+ 2715us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.82.67:46443: BlockingRecv error: recv got EOF from 127.15.82.67:46443 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Sep 29 10:33:54 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.65@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:33:54 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.65@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:33:54.182173 24887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:54.176835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:33713 (local address 127.15.82.65:57381)
0929 10:33:54.177114 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:54.177128 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:54.177210 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:54.177454 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:54.177457 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:54.177634 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:54.177737 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:54.177742 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:54.178510 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:54.178512 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:54.179065 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:54.179072 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:54.179495 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:54.179498 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:54.179558 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:54.180121 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:54.180133 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:54.181989 (+ 1856us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.82.126:33713: Server oryx/127.15.82.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":106,"threads_started":1}
W20260929 10:33:54.182302 24888 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:33:54.176929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.65:57381 (local address 127.15.82.126:33713)
0929 10:33:54.177158 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:54.177161 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:54.177174 (+ 13us) server_negotiation.cc:407] Connection header received
0929 10:33:54.177290 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:54.177292 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:54.177331 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:54.177413 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:54.177840 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:54.178365 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:54.179164 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:54.179389 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:54.179477 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:54.182150 (+ 2673us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.82.65:57381: BlockingRecv error: recv got EOF from 127.15.82.65:57381 (error 108)
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":86,"threads_started":1}
Sep 29 10:33:54 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.66@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:33:54 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.66@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:33:54.351962 24889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:54.346933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:33713 (local address 127.15.82.66:54325)
0929 10:33:54.347200 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:54.347213 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:54.347300 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:54.347513 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:54.347515 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:54.347691 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:54.347787 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:54.347791 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:54.348479 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:54.348481 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:54.349023 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:54.349029 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:54.349461 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:54.349464 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:54.349521 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:54.349992 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:54.350002 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:54.351848 (+ 1846us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.82.126:33713: Server oryx/127.15.82.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":82,"threads_started":1}
W20260929 10:33:54.352134 24888 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:33:54.346973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:54325 (local address 127.15.82.126:33713)
0929 10:33:54.347100 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:54.347104 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:54.347275 (+ 171us) server_negotiation.cc:407] Connection header received
0929 10:33:54.347363 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:54.347365 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:54.347405 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:54.347476 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:54.347878 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:54.348390 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:54.349135 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:54.349356 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:54.349448 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:54.351993 (+ 2545us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.82.66:54325: BlockingRecv error: recv got EOF from 127.15.82.66:54325 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Sep 29 10:33:54 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.67@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:33:54 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.67@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:33:54.539724 24890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:54.534329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:33713 (local address 127.15.82.67:34219)
0929 10:33:54.534665 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:54.534678 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:54.534804 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:54.535040 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:54.535041 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:54.535212 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:54.535309 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:54.535314 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:54.536120 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:54.536124 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:54.536687 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:54.536697 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:54.537109 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:54.537112 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:54.537174 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:54.537635 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:54.537646 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:54.539551 (+ 1905us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.82.126:33713: Server oryx/127.15.82.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":98,"threads_started":1}
W20260929 10:33:54.539784 24888 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:33:54.534468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.67:34219 (local address 127.15.82.126:33713)
0929 10:33:54.534611 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:54.534616 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:54.534781 (+ 165us) server_negotiation.cc:407] Connection header received
0929 10:33:54.534891 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:54.534894 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:54.534940 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:54.535007 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:54.535406 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:54.535925 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:54.536790 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:54.537011 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:54.537084 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:54.539677 (+ 2593us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.82.67:34219: BlockingRecv error: recv got EOF from 127.15.82.67:34219 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Sep 29 10:33:55 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.65@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:33:55 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.65@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:33:55.188651 24891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:55.183241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:33713 (local address 127.15.82.65:57435)
0929 10:33:55.183535 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:55.183550 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:55.183649 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:55.183942 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:55.183944 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:55.184120 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:55.184215 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:55.184219 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:55.184994 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:55.184997 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:55.185546 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:55.185553 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:55.186055 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:55.186058 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:55.186120 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:55.186738 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:55.186750 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:55.188535 (+ 1785us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.82.126:33713: Server oryx/127.15.82.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":106,"threads_started":1}
W20260929 10:33:55.188890 24892 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:33:55.183308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.65:57435 (local address 127.15.82.126:33713)
0929 10:33:55.183565 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:55.183569 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:55.183607 (+ 38us) server_negotiation.cc:407] Connection header received
0929 10:33:55.183775 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:55.183777 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:55.183822 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:55.183903 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:55.184320 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:55.184885 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:55.185687 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:55.185951 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:55.186035 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:55.188682 (+ 2647us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.82.65:57435: BlockingRecv error: recv got EOF from 127.15.82.65:57435 (error 108)
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":64,"threads_started":1}
Sep 29 10:33:55 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.66@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:33:55 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.66@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:33:55.358973 24893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:55.352758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:33713 (local address 127.15.82.66:43443)
0929 10:33:55.353075 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:55.353088 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:55.353222 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:55.353501 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:55.353504 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:55.353719 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:55.353814 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:55.353819 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:55.354661 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:55.354666 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:55.355225 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:55.355233 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:55.355760 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:55.355763 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:55.355853 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:55.356397 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:55.356409 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:55.358809 (+ 2400us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.82.126:33713: Server oryx/127.15.82.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":113,"threads_started":1}
W20260929 10:33:55.359068 24892 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:33:55.352826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:43443 (local address 127.15.82.126:33713)
0929 10:33:55.353012 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:55.353017 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:55.353161 (+ 144us) server_negotiation.cc:407] Connection header received
0929 10:33:55.353323 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:55.353327 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:55.353383 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:55.353472 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:55.353944 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:55.354551 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:55.355348 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:55.355644 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:55.355753 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:55.358950 (+ 3197us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.82.66:43443: BlockingRecv error: recv got EOF from 127.15.82.66:43443 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Sep 29 10:33:55 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.67@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:33:55 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.67@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:33:55.546482 24894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:55.540680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:33713 (local address 127.15.82.67:52521)
0929 10:33:55.541020 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:55.541036 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:55.541187 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:55.541470 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:55.541473 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:55.541685 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:55.541781 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:55.541787 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:55.542631 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:55.542636 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:55.543202 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:55.543210 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:55.543730 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:55.543733 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:55.543826 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:55.544301 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:55.544313 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:55.546255 (+ 1942us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.82.126:33713: Server oryx/127.15.82.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":86,"threads_started":1}
W20260929 10:33:55.546581 24892 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:33:55.540844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.67:52521 (local address 127.15.82.126:33713)
0929 10:33:55.541020 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:55.541026 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:55.541123 (+ 97us) server_negotiation.cc:407] Connection header received
0929 10:33:55.541292 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:55.541295 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:55.541352 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:55.541448 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:55.541919 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:55.542522 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:55.543325 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:55.543568 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:55.543714 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:55.546450 (+ 2736us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.82.67:52521: BlockingRecv error: recv got EOF from 127.15.82.67:52521 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Sep 29 10:33:56 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.65@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:33:56 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.65@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:33:56.194700 24895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:56.189371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:33713 (local address 127.15.82.65:43531)
0929 10:33:56.189643 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:56.189657 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:56.189746 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:56.190027 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:56.190029 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:56.190205 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:56.190304 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:56.190310 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:56.191057 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:56.191060 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:56.191635 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:56.191644 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:56.192061 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:56.192064 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:56.192127 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:56.192700 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:56.192712 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:56.194584 (+ 1872us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.82.126:33713: Server oryx/127.15.82.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":96,"threads_started":1}
W20260929 10:33:56.194921 24896 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:33:56.189476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.65:43531 (local address 127.15.82.126:33713)
0929 10:33:56.189708 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:56.189712 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:56.189725 (+ 13us) server_negotiation.cc:407] Connection header received
0929 10:33:56.189841 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:56.189844 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:56.189896 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:56.189974 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:56.190431 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:56.190965 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:56.191735 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:56.191961 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:56.192035 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:56.194730 (+ 2695us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.82.65:43531: BlockingRecv error: recv got EOF from 127.15.82.65:43531 (error 108)
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":79,"threads_started":1}
Sep 29 10:33:56 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.66@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:33:56 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.66@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:33:56.368374 24897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:56.359809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:33713 (local address 127.15.82.66:59887)
0929 10:33:56.360147 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:56.360160 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:56.360260 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:56.360531 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:56.360534 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:56.360742 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:56.360882 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:56.360893 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:56.362833 (+ 1940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:56.362838 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:56.363777 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:56.363787 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:56.364858 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:56.364864 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:56.364976 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:56.365514 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:56.365530 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:56.368162 (+ 2632us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.82.126:33713: Server oryx/127.15.82.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":88,"threads_started":1}
W20260929 10:33:56.368708 24896 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:33:56.359933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:59887 (local address 127.15.82.126:33713)
0929 10:33:56.360072 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:56.360076 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:56.360237 (+ 161us) server_negotiation.cc:407] Connection header received
0929 10:33:56.360373 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:56.360376 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:56.360430 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:56.360508 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:56.361030 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:56.362548 (+ 1518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:56.363902 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:56.364680 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:56.364812 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:56.368568 (+ 3756us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.82.66:59887: BlockingRecv error: recv got EOF from 127.15.82.66:59887 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Sep 29 10:33:56 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.67@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:33:56 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.67@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:33:56.552562 24898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:56.547376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:33713 (local address 127.15.82.67:32919)
0929 10:33:56.547648 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:56.547662 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:56.547758 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:56.548007 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:56.548009 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:56.548190 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:56.548286 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:56.548291 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:56.549008 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:56.549011 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:56.549564 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:56.549572 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:56.549974 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:56.549977 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:56.550040 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:56.550641 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:56.550653 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:56.552444 (+ 1791us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.82.126:33713: Server oryx/127.15.82.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":108,"threads_started":1}
W20260929 10:33:56.552774 24896 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:33:56.547413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.67:32919 (local address 127.15.82.126:33713)
0929 10:33:56.547572 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:56.547576 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:56.547754 (+ 178us) server_negotiation.cc:407] Connection header received
0929 10:33:56.547828 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:56.547830 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:56.547883 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:56.547959 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:56.548393 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:56.548907 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:56.549666 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:56.549876 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:56.549950 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:56.552588 (+ 2638us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.82.67:32919: BlockingRecv error: recv got EOF from 127.15.82.67:32919 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Sep 29 10:33:57 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.65@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:33:57 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.65@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:33:57.205885 24899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:57.195640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:33713 (local address 127.15.82.65:60681)
0929 10:33:57.195973 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:57.195989 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:57.196102 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:57.196415 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:57.196418 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:57.196667 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:57.196799 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:57.196809 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:57.197805 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:57.197808 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:57.198358 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:57.198365 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:57.198882 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:57.198885 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:57.198950 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:57.203639 (+ 4689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:57.203654 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:57.205754 (+ 2100us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.82.126:33713: Server oryx/127.15.82.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":107,"threads_started":1}
W20260929 10:33:57.206089 24900 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:33:57.195732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.65:60681 (local address 127.15.82.126:33713)
0929 10:33:57.196018 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:57.196022 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:57.196041 (+ 19us) server_negotiation.cc:407] Connection header received
0929 10:33:57.196211 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:57.196214 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:57.196271 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:57.196389 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:57.196939 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:57.197702 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:57.198525 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:57.198771 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:57.198861 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:57.205933 (+ 7072us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.82.65:60681: BlockingRecv error: recv got EOF from 127.15.82.65:60681 (error 108)
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":122,"threads_started":1}
Sep 29 10:33:57 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.66@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:33:57 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.66@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:33:57.375834 24901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:57.369434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:33713 (local address 127.15.82.66:47061)
0929 10:33:57.369747 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:57.369767 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:57.369895 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:57.370259 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:57.370262 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:57.370567 (+ 305us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:57.370717 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:57.370728 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:57.371658 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:57.371664 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:57.372636 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:57.372646 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:57.373133 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:57.373138 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:57.373227 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:57.373814 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:57.373836 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:57.375689 (+ 1853us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.82.126:33713: Server oryx/127.15.82.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":84,"threads_started":1}
W20260929 10:33:57.376050 24900 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:33:57.369609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:47061 (local address 127.15.82.126:33713)
0929 10:33:57.369764 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:57.369767 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:57.369838 (+ 71us) server_negotiation.cc:407] Connection header received
0929 10:33:57.370030 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:57.370033 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:57.370089 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:57.370194 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:57.370853 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:57.371524 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:57.372766 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:57.373016 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:57.373126 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:57.375870 (+ 2744us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.82.66:47061: BlockingRecv error: recv got EOF from 127.15.82.66:47061 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Sep 29 10:33:57 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.67@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:33:57 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.67@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:33:57.559614 24902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:57.553471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:33713 (local address 127.15.82.67:51213)
0929 10:33:57.553776 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:57.553791 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:57.553935 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:57.554222 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:57.554225 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:57.554519 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:57.554617 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:57.554624 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:57.555396 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:57.555402 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:57.555976 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:57.555984 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:57.556442 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:57.556445 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:57.556532 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:57.557103 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:57.557118 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:57.559445 (+ 2327us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.82.126:33713: Server oryx/127.15.82.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":86,"threads_started":1}
W20260929 10:33:57.559677 24900 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:33:57.553548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.67:51213 (local address 127.15.82.126:33713)
0929 10:33:57.553725 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:57.553730 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:57.553878 (+ 148us) server_negotiation.cc:407] Connection header received
0929 10:33:57.554057 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:57.554060 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:57.554106 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:57.554194 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:57.554741 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:57.555283 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:57.556097 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:57.556328 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:57.556438 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:57.559567 (+ 3129us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.82.67:51213: BlockingRecv error: recv got EOF from 127.15.82.67:51213 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Sep 29 10:33:58 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.65@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:33:58 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.65@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:33:58.212426 24903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:58.206724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:33713 (local address 127.15.82.65:55565)
0929 10:33:58.207003 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:58.207015 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:58.207107 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:58.207509 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:58.207511 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:58.207691 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:58.207788 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:58.207793 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:58.208503 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:58.208505 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:58.209041 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:58.209047 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:58.209451 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:58.209454 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:58.209512 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:58.210080 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:58.210092 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:58.212247 (+ 2155us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.82.126:33713: Server oryx/127.15.82.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":105,"threads_started":1}
W20260929 10:33:58.212656 24904 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:33:58.206937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.65:55565 (local address 127.15.82.126:33713)
0929 10:33:58.207278 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:58.207282 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:58.207294 (+ 12us) server_negotiation.cc:407] Connection header received
0929 10:33:58.207330 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:58.207332 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:58.207387 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:58.207468 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:58.207889 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:58.208415 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:58.209134 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:58.209359 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:58.209433 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:58.212402 (+ 2969us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.82.65:55565: BlockingRecv error: recv got EOF from 127.15.82.65:55565 (error 108)
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":113,"threads_started":1}
Sep 29 10:33:58 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.66@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:33:58 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.66@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:33:58.382108 24905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:58.376602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:33713 (local address 127.15.82.66:49373)
0929 10:33:58.376911 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:58.376925 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:58.377019 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:58.377236 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:58.377238 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:58.377417 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:58.377509 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:58.377514 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:58.378219 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:58.378221 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:58.378845 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:58.378853 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:58.379259 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:58.379261 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:58.379320 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:58.379795 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:58.379806 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:58.381992 (+ 2186us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.82.126:33713: Server oryx/127.15.82.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":118,"threads_started":1}
W20260929 10:33:58.382283 24904 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:33:58.376657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:49373 (local address 127.15.82.126:33713)
0929 10:33:58.376812 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:58.376817 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:58.376995 (+ 178us) server_negotiation.cc:407] Connection header received
0929 10:33:58.377095 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:58.377097 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:58.377143 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:58.377201 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:58.377603 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:58.378132 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:58.378943 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:58.379155 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:58.379229 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:58.382126 (+ 2897us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.82.66:49373: BlockingRecv error: recv got EOF from 127.15.82.66:49373 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Sep 29 10:33:58 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.67@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:33:58 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.67@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:33:58.565850 24906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:58.560550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:33713 (local address 127.15.82.67:59621)
0929 10:33:58.560850 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:58.560864 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:58.560963 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:58.561198 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:58.561200 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:58.561392 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:58.561487 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:58.561493 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:58.562193 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:58.562196 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:58.562821 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:58.562831 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:58.563232 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:58.563234 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:58.563296 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:58.563778 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:58.563789 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:58.565717 (+ 1928us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.82.126:33713: Server oryx/127.15.82.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":116,"threads_started":1}
W20260929 10:33:58.566013 24904 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:33:58.560606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.67:59621 (local address 127.15.82.126:33713)
0929 10:33:58.560757 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:58.560762 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:58.560938 (+ 176us) server_negotiation.cc:407] Connection header received
0929 10:33:58.561044 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:58.561046 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:58.561094 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:58.561161 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:58.561587 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:58.562104 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:58.562927 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:58.563140 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:58.563215 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:58.565858 (+ 2643us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.82.67:59621: BlockingRecv error: recv got EOF from 127.15.82.67:59621 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Sep 29 10:33:59 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.65@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:33:59 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.65@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:33:59.218442 24907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:59.213263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:33713 (local address 127.15.82.65:50639)
0929 10:33:59.213523 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:59.213536 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:59.213616 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:59.213847 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:59.213849 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:59.214030 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:59.214124 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:59.214129 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:59.214850 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:59.214852 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:59.215399 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:59.215406 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:59.215816 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:59.215821 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:59.215884 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:59.216363 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:59.216373 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:59.218230 (+ 1857us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.82.126:33713: Server oryx/127.15.82.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":92,"threads_started":1}
W20260929 10:33:59.218505 24908 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:33:59.213365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.65:50639 (local address 127.15.82.126:33713)
0929 10:33:59.213592 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:59.213595 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:59.213608 (+ 13us) server_negotiation.cc:407] Connection header received
0929 10:33:59.213699 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:59.213701 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:59.213742 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:59.213810 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:59.214221 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:59.214767 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:59.215491 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:59.215710 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:59.215784 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:59.218357 (+ 2573us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.82.65:50639: BlockingRecv error: recv got EOF from 127.15.82.65:50639 (error 108)
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":80,"threads_started":1}
Sep 29 10:33:59 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.66@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:33:59 dist-test-slave-vg89 krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.82.66@KRBTEST.COM for oryx/127.15.82.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:33:59.388844 24909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:59.383171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:33713 (local address 127.15.82.66:47879)
0929 10:33:59.383494 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:59.383509 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:59.383614 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:59.383850 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:59.383852 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:59.384093 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:59.384214 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:59.384220 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:59.385126 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:59.385131 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:59.385831 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:59.385840 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:59.386273 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:59.386276 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:59.386348 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:59.386903 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:33:59.386915 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:59.388705 (+ 1790us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.82.126:33713: Server oryx/127.15.82.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":115,"threads_started":1}
W20260929 10:33:59.388979 24908 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:33:59.383265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:47879 (local address 127.15.82.126:33713)
0929 10:33:59.383448 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:59.383454 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:59.383568 (+ 114us) server_negotiation.cc:407] Connection header received
0929 10:33:59.383687 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:59.383689 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:59.383734 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:59.383808 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:59.384358 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:59.385000 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:59.385932 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:59.386153 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:59.386238 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:59.388856 (+ 2618us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.82.66:47879: BlockingRecv error: recv got EOF from 127.15.82.66:47879 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260929 10:33:59.455090 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 24433
I20260929 10:33:59.461742 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 24585
I20260929 10:33:59.467552 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 24717
I20260929 10:33:59.472978 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 24363
2026-09-29T10:33:59Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17938 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-29T10:33:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:33:59Z Disabled control of system clock
I20260929 10:33:59.504508 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790677981592896-15689-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790677981592896-15689-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790677981592896-15689-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:34795
--webserver_interface=127.15.82.126
--webserver_port=0
--builtin_ntp_servers=127.15.82.84:36351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.82.126:34795
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260929 10:33:59.627046 24913 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:33:59.627321 24913 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:33:59.627394 24913 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:33:59.631464 24913 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:33:59.631531 24913 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:33:59.631551 24913 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:33:59.631606 24913 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:33:59.631654 24913 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:33:59.636477 24913 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:36351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790677981592896-15689-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790677981592896-15689-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.82.126:34795
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:34795
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.82.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24913
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790677981592896-15689-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:33:59.637610 24913 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:33:59.639168 24913 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 10:33:59.647876 24913 server_base.cc:1034] running on GCE node
W20260929 10:33:59.647826 24919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:59.647863 24921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:59.647826 24918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:59.648612 24913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:33:59.649662 24913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:33:59.650838 24913 hybrid_clock.cc:648] HybridClock initialized: now 1790678039650823 us; error 27 us; skew 500 ppm
I20260929 10:33:59.652705 24913 webserver.cc:460] Webserver started at http://127.15.82.126:37389/ using document root <none> and password file <none>
I20260929 10:33:59.653288 24913 fs_manager.cc:359] Metadata directory not provided
I20260929 10:33:59.653378 24913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:33:59.653607 24913 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:33:59.655699 24913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790677981592896-15689-0/minicluster-data/master-0/data/instance:
uuid: "b8b18fac8d09422f96a1ed69886b6e5c"
format_stamp: "Formatted at 2026-09-29 10:33:59 on dist-test-slave-vg89"
server_key: "e9e9abd4e52817096b0314b6f43a0186"
server_key_iv: "1f8559c6b32696d97c84689ffca125a4"
server_key_version: "encryptionkey@0"
I20260929 10:33:59.656253 24913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790677981592896-15689-0/minicluster-data/master-0/wal/instance:
uuid: "b8b18fac8d09422f96a1ed69886b6e5c"
format_stamp: "Formatted at 2026-09-29 10:33:59 on dist-test-slave-vg89"
server_key: "e9e9abd4e52817096b0314b6f43a0186"
server_key_iv: "1f8559c6b32696d97c84689ffca125a4"
server_key_version: "encryptionkey@0"
I20260929 10:33:59.659848 24913 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260929 10:33:59.662113 24927 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:59.663314 24913 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 10:33:59.663465 24913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790677981592896-15689-0/minicluster-data/master-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790677981592896-15689-0/minicluster-data/master-0/wal
uuid: "b8b18fac8d09422f96a1ed69886b6e5c"
format_stamp: "Formatted at 2026-09-29 10:33:59 on dist-test-slave-vg89"
server_key: "e9e9abd4e52817096b0314b6f43a0186"
server_key_iv: "1f8559c6b32696d97c84689ffca125a4"
server_key_version: "encryptionkey@0"
I20260929 10:33:59.663576 24913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790677981592896-15689-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790677981592896-15689-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790677981592896-15689-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:33:59.682276 24913 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:33:59.683053 24913 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:33:59.683259 24913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:33:59.689965 24913 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.126:34795
I20260929 10:33:59.689976 24979 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.126:34795 every 8 connection(s)
I20260929 10:33:59.690938 24913 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
I20260929 10:33:59.694077 24980 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:33:59.699527 24980 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8b18fac8d09422f96a1ed69886b6e5c: Bootstrap starting.
I20260929 10:33:59.700773 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 24913
I20260929 10:33:59.700904 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790677981592896-15689-0/minicluster-data/master-0/wal/instance
I20260929 10:33:59.701179 15689 external_mini_cluster.cc:1428] Setting key c3c381fecf023d2341293e9cde102bac
I20260929 10:33:59.702100 24980 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b8b18fac8d09422f96a1ed69886b6e5c: Neither blocks nor log segments found. Creating new log.
I20260929 10:33:59.703007 24980 log.cc:824] T 00000000000000000000000000000000 P b8b18fac8d09422f96a1ed69886b6e5c: Log is configured to *not* fsync() on all Append() calls
I20260929 10:33:59.704823 24980 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8b18fac8d09422f96a1ed69886b6e5c: No bootstrap required, opened a new log
I20260929 10:33:59.708246 24980 raft_consensus.cc:348] T 00000000000000000000000000000000 P b8b18fac8d09422f96a1ed69886b6e5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8b18fac8d09422f96a1ed69886b6e5c" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 34795 } }
I20260929 10:33:59.708500 24980 raft_consensus.cc:374] T 00000000000000000000000000000000 P b8b18fac8d09422f96a1ed69886b6e5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:33:59.708581 24980 raft_consensus.cc:729] T 00000000000000000000000000000000 P b8b18fac8d09422f96a1ed69886b6e5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8b18fac8d09422f96a1ed69886b6e5c, State: Initialized, Role: FOLLOWER
I20260929 10:33:59.709043 24980 consensus_queue.cc:260] T 00000000000000000000000000000000 P b8b18fac8d09422f96a1ed69886b6e5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8b18fac8d09422f96a1ed69886b6e5c" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 34795 } }
I20260929 10:33:59.709193 24980 raft_consensus.cc:388] T 00000000000000000000000000000000 P b8b18fac8d09422f96a1ed69886b6e5c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:33:59.709275 24980 raft_consensus.cc:482] T 00000000000000000000000000000000 P b8b18fac8d09422f96a1ed69886b6e5c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:33:59.709398 24980 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b8b18fac8d09422f96a1ed69886b6e5c [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:33:59.710289 24983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:59.702328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41140 (local address 127.15.82.126:34795)
0929 10:33:59.703002 (+ 674us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:59.703015 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:59.703049 (+ 34us) server_negotiation.cc:407] Connection header received
0929 10:33:59.703709 (+ 660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:59.703726 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:59.704063 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:59.704398 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:59.704813 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:59.705846 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:59.706838 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:59.707306 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:59.707480 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:59.708324 (+ 844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:59.708330 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:59.708345 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:33:59.708818 (+ 473us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:59.708885 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:33:59.708906 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:59.709009 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:59.709129 (+ 120us) server_negotiation.cc:299] Negotiation successful
0929 10:33:59.709567 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":129,"threads_started":1}
I20260929 10:33:59.710296 24980 raft_consensus.cc:504] T 00000000000000000000000000000000 P b8b18fac8d09422f96a1ed69886b6e5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8b18fac8d09422f96a1ed69886b6e5c" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 34795 } }
I20260929 10:33:59.710793 24980 leader_election.cc:302] T 00000000000000000000000000000000 P b8b18fac8d09422f96a1ed69886b6e5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b8b18fac8d09422f96a1ed69886b6e5c; no voters:
I20260929 10:33:59.711115 24980 leader_election.cc:288] T 00000000000000000000000000000000 P b8b18fac8d09422f96a1ed69886b6e5c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:33:59.711252 24985 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b8b18fac8d09422f96a1ed69886b6e5c [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:33:59.711474 24985 raft_consensus.cc:686] T 00000000000000000000000000000000 P b8b18fac8d09422f96a1ed69886b6e5c [term 1 LEADER]: Becoming Leader. State: Replica: b8b18fac8d09422f96a1ed69886b6e5c, State: Running, Role: LEADER
I20260929 10:33:59.711826 24985 consensus_queue.cc:237] T 00000000000000000000000000000000 P b8b18fac8d09422f96a1ed69886b6e5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8b18fac8d09422f96a1ed69886b6e5c" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 34795 } }
I20260929 10:33:59.712293 24980 sys_catalog.cc:564] T 00000000000000000000000000000000 P b8b18fac8d09422f96a1ed69886b6e5c [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:33:59.713073 24987 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8b18fac8d09422f96a1ed69886b6e5c [sys.catalog]: SysCatalogTable state changed. Reason: New leader b8b18fac8d09422f96a1ed69886b6e5c. Latest consensus state: current_term: 1 leader_uuid: "b8b18fac8d09422f96a1ed69886b6e5c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8b18fac8d09422f96a1ed69886b6e5c" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 34795 } } }
I20260929 10:33:59.713203 24987 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8b18fac8d09422f96a1ed69886b6e5c [sys.catalog]: This master's current role is: LEADER
I20260929 10:33:59.713433 24986 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8b18fac8d09422f96a1ed69886b6e5c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b8b18fac8d09422f96a1ed69886b6e5c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8b18fac8d09422f96a1ed69886b6e5c" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 34795 } } }
I20260929 10:33:59.713534 24986 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8b18fac8d09422f96a1ed69886b6e5c [sys.catalog]: This master's current role is: LEADER
I20260929 10:33:59.713795 24994 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:33:59.716526 24994 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:33:59.721654 24994 catalog_manager.cc:1347] Generated new cluster ID: ef1d8b61e15643a2ac79fad135ba21e4
I20260929 10:33:59.721722 24994 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:33:59.762163 24994 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:33:59.763464 24994 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:33:59.771375 24994 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b8b18fac8d09422f96a1ed69886b6e5c: Generated new TSK 0
I20260929 10:33:59.772176 24994 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 10:33:59.782377 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.65:0
--local_ip_for_outbound_sockets=127.15.82.65
--webserver_interface=127.15.82.65
--webserver_port=0
--tserver_master_addrs=127.15.82.126:34795
--builtin_ntp_servers=127.15.82.84:36351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260929 10:33:59.894758 25004 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:33:59.895069 25004 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:33:59.895156 25004 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:33:59.899178 25004 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:33:59.899289 25004 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:33:59.899405 25004 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.65
I20260929 10:33:59.904090 25004 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:36351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.82.65
--webserver_port=0
--tserver_master_addrs=127.15.82.126:34795
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25004
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.65
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:33:59.905191 25004 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:33:59.906632 25004 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:33:59.916906 25009 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:59.916924 25010 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:33:59.916918 25012 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:33:59.917018 25004 server_base.cc:1034] running on GCE node
I20260929 10:33:59.917565 25004 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:33:59.918282 25004 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:33:59.919446 25004 hybrid_clock.cc:648] HybridClock initialized: now 1790678039919410 us; error 50 us; skew 500 ppm
I20260929 10:33:59.921279 25004 webserver.cc:460] Webserver started at http://127.15.82.65:46339/ using document root <none> and password file <none>
I20260929 10:33:59.921883 25004 fs_manager.cc:359] Metadata directory not provided
I20260929 10:33:59.921972 25004 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:33:59.922192 25004 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:33:59.924368 25004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-0/data/instance:
uuid: "c9781be62742417091990697023dc1da"
format_stamp: "Formatted at 2026-09-29 10:33:59 on dist-test-slave-vg89"
server_key: "07c58a7762eb01347df3263c9d24d615"
server_key_iv: "0a583cc6958c3492bba74c27534d7c95"
server_key_version: "encryptionkey@0"
I20260929 10:33:59.924952 25004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-0/wal/instance:
uuid: "c9781be62742417091990697023dc1da"
format_stamp: "Formatted at 2026-09-29 10:33:59 on dist-test-slave-vg89"
server_key: "07c58a7762eb01347df3263c9d24d615"
server_key_iv: "0a583cc6958c3492bba74c27534d7c95"
server_key_version: "encryptionkey@0"
I20260929 10:33:59.928786 25004 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 10:33:59.931469 25018 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:59.932725 25004 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 10:33:59.932873 25004 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-0/wal
uuid: "c9781be62742417091990697023dc1da"
format_stamp: "Formatted at 2026-09-29 10:33:59 on dist-test-slave-vg89"
server_key: "07c58a7762eb01347df3263c9d24d615"
server_key_iv: "0a583cc6958c3492bba74c27534d7c95"
server_key_version: "encryptionkey@0"
I20260929 10:33:59.932993 25004 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:33:59.950042 25004 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:33:59.950913 25004 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:33:59.951120 25004 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:33:59.951736 25004 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:33:59.952720 25004 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:33:59.952790 25004 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:59.952884 25004 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:33:59.952930 25004 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:33:59.961994 25004 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.65:38783
I20260929 10:33:59.962019 25131 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.65:38783 every 8 connection(s)
I20260929 10:33:59.962989 25004 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
I20260929 10:33:59.969203 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 25004
I20260929 10:33:59.969349 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-0/wal/instance
I20260929 10:33:59.969642 15689 external_mini_cluster.cc:1428] Setting key 2defa05d48c12b1e57d90c16b70efc3f
I20260929 10:33:59.971621 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.66:0
--local_ip_for_outbound_sockets=127.15.82.66
--webserver_interface=127.15.82.66
--webserver_port=0
--tserver_master_addrs=127.15.82.126:34795
--builtin_ntp_servers=127.15.82.84:36351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260929 10:33:59.972939 24983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:59.964888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.65:57565 (local address 127.15.82.126:34795)
0929 10:33:59.965035 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0929 10:33:59.965039 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:33:59.966134 (+ 1095us) server_negotiation.cc:407] Connection header received
0929 10:33:59.967726 (+ 1592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:33:59.967729 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:33:59.967792 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:33:59.967905 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:33:59.968898 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:59.969517 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:59.970914 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:33:59.971144 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:33:59.971255 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:59.972439 (+ 1184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:33:59.972442 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:33:59.972445 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:33:59.972458 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:33:59.972482 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:33:59.972489 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:33:59.972565 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0929 10:33:59.972724 (+ 159us) server_negotiation.cc:299] Negotiation successful
0929 10:33:59.972787 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260929 10:33:59.973886 25134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:33:59.965314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:34795 (local address 127.15.82.65:57565)
0929 10:33:59.965974 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0929 10:33:59.966017 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 10:33:59.967476 (+ 1459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:33:59.968111 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:33:59.968121 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:33:59.968222 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:33:59.968736 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:59.968749 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:59.969638 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:59.969641 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:59.970779 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:33:59.970789 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:33:59.971291 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:33:59.971295 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:33:59.971470 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:33:59.972145 (+ 675us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 10:33:59.972147 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:33:59.972189 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 10:33:59.972256 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:33:59.972567 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:33:59.972580 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:33:59.972588 (+ 8us) client_negotiation.cc:769] Sending connection context
0929 10:33:59.972722 (+ 134us) client_negotiation.cc:240] Negotiation successful
0929 10:33:59.973033 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":117,"threads_started":1}
I20260929 10:33:59.975206 25132 heartbeater.cc:347] Connected to a master server at 127.15.82.126:34795
I20260929 10:33:59.975472 25132 heartbeater.cc:464] Registering TS with master...
I20260929 10:33:59.976019 25132 heartbeater.cc:511] Master 127.15.82.126:34795 requested a full tablet report, sending...
I20260929 10:33:59.977573 24944 ts_manager.cc:194] Registered new tserver with Master: c9781be62742417091990697023dc1da (127.15.82.65:38783)
I20260929 10:33:59.978840 24944 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.82.65:57565
W20260929 10:34:00.087451 25135 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:34:00.087729 25135 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:34:00.087769 25135 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:34:00.091799 25135 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:34:00.091854 25135 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:34:00.091910 25135 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.66
I20260929 10:34:00.096496 25135 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:36351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.82.66
--webserver_port=0
--tserver_master_addrs=127.15.82.126:34795
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25135
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.66
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:34:00.097546 25135 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:34:00.098976 25135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:34:00.108675 25140 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:34:00.108850 25135 server_base.cc:1034] running on GCE node
W20260929 10:34:00.108865 25143 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:34:00.108716 25141 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:34:00.109412 25135 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:34:00.110049 25135 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:34:00.111243 25135 hybrid_clock.cc:648] HybridClock initialized: now 1790678040111214 us; error 48 us; skew 500 ppm
I20260929 10:34:00.113304 25135 webserver.cc:460] Webserver started at http://127.15.82.66:38221/ using document root <none> and password file <none>
I20260929 10:34:00.113904 25135 fs_manager.cc:359] Metadata directory not provided
I20260929 10:34:00.113988 25135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:34:00.114190 25135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:34:00.116214 25135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-1/data/instance:
uuid: "a99777f9f2444b7da231b0ebec34f5fb"
format_stamp: "Formatted at 2026-09-29 10:34:00 on dist-test-slave-vg89"
server_key: "3383322f0d6d5df23ac836fcf9bb01ad"
server_key_iv: "f12b4e76ff6f526dc8ee0af9a87bed48"
server_key_version: "encryptionkey@0"
I20260929 10:34:00.116767 25135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-1/wal/instance:
uuid: "a99777f9f2444b7da231b0ebec34f5fb"
format_stamp: "Formatted at 2026-09-29 10:34:00 on dist-test-slave-vg89"
server_key: "3383322f0d6d5df23ac836fcf9bb01ad"
server_key_iv: "f12b4e76ff6f526dc8ee0af9a87bed48"
server_key_version: "encryptionkey@0"
I20260929 10:34:00.120368 25135 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260929 10:34:00.122881 25149 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:34:00.124136 25135 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 10:34:00.124281 25135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-1/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-1/wal
uuid: "a99777f9f2444b7da231b0ebec34f5fb"
format_stamp: "Formatted at 2026-09-29 10:34:00 on dist-test-slave-vg89"
server_key: "3383322f0d6d5df23ac836fcf9bb01ad"
server_key_iv: "f12b4e76ff6f526dc8ee0af9a87bed48"
server_key_version: "encryptionkey@0"
I20260929 10:34:00.124394 25135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:34:00.136540 25135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:34:00.137252 25135 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:34:00.137452 25135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:34:00.138031 25135 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:34:00.139020 25135 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:34:00.139088 25135 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:34:00.139153 25135 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:34:00.139199 25135 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:34:00.148013 25135 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.66:44391
I20260929 10:34:00.148034 25262 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.66:44391 every 8 connection(s)
I20260929 10:34:00.148964 25135 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
I20260929 10:34:00.158186 24983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:34:00.150791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:38497 (local address 127.15.82.126:34795)
0929 10:34:00.150947 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0929 10:34:00.150951 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:34:00.152053 (+ 1102us) server_negotiation.cc:407] Connection header received
0929 10:34:00.153455 (+ 1402us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:34:00.153459 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:34:00.153519 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:34:00.153619 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:34:00.154584 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:34:00.155126 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:34:00.156278 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:34:00.156511 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:34:00.156620 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:34:00.157696 (+ 1076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:34:00.157699 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:34:00.157703 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:34:00.157720 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:34:00.157738 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:34:00.157746 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:34:00.157817 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0929 10:34:00.157975 (+ 158us) server_negotiation.cc:299] Negotiation successful
0929 10:34:00.158037 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260929 10:34:00.158663 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 25135
I20260929 10:34:00.158803 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-1/wal/instance
I20260929 10:34:00.159108 15689 external_mini_cluster.cc:1428] Setting key 19a91805274777d810e21cd6d3912b87
I20260929 10:34:00.159143 25265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:34:00.151262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:34795 (local address 127.15.82.66:38497)
0929 10:34:00.151912 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0929 10:34:00.151953 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 10:34:00.153239 (+ 1286us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:34:00.153800 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:34:00.153812 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:34:00.153905 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:34:00.154444 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:34:00.154464 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:34:00.155251 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:34:00.155255 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:34:00.156155 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:34:00.156164 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:34:00.156661 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:34:00.156665 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:34:00.156837 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:34:00.157440 (+ 603us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 10:34:00.157442 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:34:00.157481 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 10:34:00.157543 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:34:00.157838 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:34:00.157851 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:34:00.157856 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 10:34:00.157975 (+ 119us) client_negotiation.cc:240] Negotiation successful
0929 10:34:00.158262 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":113,"threads_started":1}
I20260929 10:34:00.160390 25263 heartbeater.cc:347] Connected to a master server at 127.15.82.126:34795
I20260929 10:34:00.160621 25263 heartbeater.cc:464] Registering TS with master...
I20260929 10:34:00.161226 25263 heartbeater.cc:511] Master 127.15.82.126:34795 requested a full tablet report, sending...
I20260929 10:34:00.161386 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.67:0
--local_ip_for_outbound_sockets=127.15.82.67
--webserver_interface=127.15.82.67
--webserver_port=0
--tserver_master_addrs=127.15.82.126:34795
--builtin_ntp_servers=127.15.82.84:36351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260929 10:34:00.163069 24943 ts_manager.cc:194] Registered new tserver with Master: a99777f9f2444b7da231b0ebec34f5fb (127.15.82.66:44391)
I20260929 10:34:00.163897 24943 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.82.66:38497
W20260929 10:34:00.274286 25266 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:34:00.274600 25266 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:34:00.274639 25266 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:34:00.278580 25266 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:34:00.278627 25266 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:34:00.278680 25266 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.67
I20260929 10:34:00.283334 25266 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:36351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.82.67
--webserver_port=0
--tserver_master_addrs=127.15.82.126:34795
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25266
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.67
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:34:00.284340 25266 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:34:00.285646 25266 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:34:00.295210 25271 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:34:00.295238 25272 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:34:00.295358 25266 server_base.cc:1034] running on GCE node
W20260929 10:34:00.295228 25274 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:34:00.295876 25266 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:34:00.296496 25266 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:34:00.297638 25266 hybrid_clock.cc:648] HybridClock initialized: now 1790678040297625 us; error 25 us; skew 500 ppm
I20260929 10:34:00.299463 25266 webserver.cc:460] Webserver started at http://127.15.82.67:34631/ using document root <none> and password file <none>
I20260929 10:34:00.300033 25266 fs_manager.cc:359] Metadata directory not provided
I20260929 10:34:00.300093 25266 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:34:00.300329 25266 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:34:00.302326 25266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-2/data/instance:
uuid: "92eceb321be94dedbd6b3d134c9245ef"
format_stamp: "Formatted at 2026-09-29 10:34:00 on dist-test-slave-vg89"
server_key: "775c9810e128bb06266204d5f77cc2cb"
server_key_iv: "4674f2d89706e37dc2cd380d72a732a1"
server_key_version: "encryptionkey@0"
I20260929 10:34:00.302944 25266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-2/wal/instance:
uuid: "92eceb321be94dedbd6b3d134c9245ef"
format_stamp: "Formatted at 2026-09-29 10:34:00 on dist-test-slave-vg89"
server_key: "775c9810e128bb06266204d5f77cc2cb"
server_key_iv: "4674f2d89706e37dc2cd380d72a732a1"
server_key_version: "encryptionkey@0"
I20260929 10:34:00.306581 25266 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260929 10:34:00.309001 25280 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:34:00.310177 25266 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 10:34:00.310302 25266 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-2/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-2/wal
uuid: "92eceb321be94dedbd6b3d134c9245ef"
format_stamp: "Formatted at 2026-09-29 10:34:00 on dist-test-slave-vg89"
server_key: "775c9810e128bb06266204d5f77cc2cb"
server_key_iv: "4674f2d89706e37dc2cd380d72a732a1"
server_key_version: "encryptionkey@0"
I20260929 10:34:00.310433 25266 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:34:00.324930 25266 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:34:00.325599 25266 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:34:00.325784 25266 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:34:00.326355 25266 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:34:00.327289 25266 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:34:00.327343 25266 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:34:00.327409 25266 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:34:00.327445 25266 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:34:00.336225 25266 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.67:40567
I20260929 10:34:00.336244 25393 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.67:40567 every 8 connection(s)
I20260929 10:34:00.337208 25266 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
I20260929 10:34:00.338286 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 25266
I20260929 10:34:00.338434 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-2/wal/instance
I20260929 10:34:00.338794 15689 external_mini_cluster.cc:1428] Setting key 5d76b23acb02912c0c482effdd56e8e1
I20260929 10:34:00.347404 24983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:34:00.339373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.67:34407 (local address 127.15.82.126:34795)
0929 10:34:00.339542 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0929 10:34:00.339547 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:34:00.340776 (+ 1229us) server_negotiation.cc:407] Connection header received
0929 10:34:00.342464 (+ 1688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:34:00.342468 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:34:00.342526 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:34:00.342614 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:34:00.343535 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:34:00.344156 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:34:00.345342 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:34:00.345577 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:34:00.345703 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:34:00.346853 (+ 1150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:34:00.346864 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:34:00.346872 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:34:00.346885 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:34:00.346906 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:34:00.346915 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:34:00.347000 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0929 10:34:00.347169 (+ 169us) server_negotiation.cc:299] Negotiation successful
0929 10:34:00.347235 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260929 10:34:00.348230 25396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:34:00.339858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:34795 (local address 127.15.82.67:34407)
0929 10:34:00.340626 (+ 768us) negotiation.cc:107] Waiting for socket to connect
0929 10:34:00.340672 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0929 10:34:00.342222 (+ 1550us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:34:00.342795 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:34:00.342803 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:34:00.342890 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:34:00.343390 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:34:00.343404 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:34:00.344288 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:34:00.344292 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:34:00.345208 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:34:00.345218 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:34:00.345741 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:34:00.345745 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:34:00.345920 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:34:00.346573 (+ 653us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 10:34:00.346574 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:34:00.346616 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 10:34:00.346694 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:34:00.347001 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:34:00.347012 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:34:00.347021 (+ 9us) client_negotiation.cc:769] Sending connection context
0929 10:34:00.347168 (+ 147us) client_negotiation.cc:240] Negotiation successful
0929 10:34:00.347451 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":109,"threads_started":1}
I20260929 10:34:00.349387 25394 heartbeater.cc:347] Connected to a master server at 127.15.82.126:34795
I20260929 10:34:00.349604 25394 heartbeater.cc:464] Registering TS with master...
I20260929 10:34:00.350095 25394 heartbeater.cc:511] Master 127.15.82.126:34795 requested a full tablet report, sending...
I20260929 10:34:00.351112 24943 ts_manager.cc:194] Registered new tserver with Master: 92eceb321be94dedbd6b3d134c9245ef (127.15.82.67:40567)
I20260929 10:34:00.351715 24943 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.82.67:34407
I20260929 10:34:00.352947 15689 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260929 10:34:00.355690 24983 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:34:00.354840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41148 (local address 127.15.82.126:34795)
0929 10:34:00.354967 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0929 10:34:00.354971 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:34:00.355143 (+ 172us) server_negotiation.cc:407] Connection header received
0929 10:34:00.355235 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:34:00.355237 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:34:00.355275 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:34:00.355345 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:34:00.355567 (+ 222us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41148: BlockingRecv error: recv got EOF from 127.0.0.1:41148 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
I20260929 10:34:00.355805 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 25004
I20260929 10:34:00.362221 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 25135
I20260929 10:34:00.367713 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 25266
I20260929 10:34:00.372769 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 24913
2026-09-29T10:34:00Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (898 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-29T10:34:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:34:00Z Disabled control of system clock
I20260929 10:34:00.402369 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790677981592896-15689-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790677981592896-15689-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790677981592896-15689-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:41189
--webserver_interface=127.15.82.126
--webserver_port=0
--builtin_ntp_servers=127.15.82.84:42755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.82.126:41189
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260929 10:34:00.515355 25405 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:34:00.515622 25405 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:34:00.515659 25405 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:34:00.519752 25405 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:34:00.519809 25405 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:34:00.519827 25405 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:34:00.519850 25405 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:34:00.520069 25405 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:34:00.525257 25405 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:42755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790677981592896-15689-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790677981592896-15689-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.82.126:41189
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:41189
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.82.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25405
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790677981592896-15689-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:34:00.526324 25405 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:34:00.527904 25405 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:34:00.536410 25411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:34:00.536410 25413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:34:00.536432 25410 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:34:00.536686 25405 server_base.cc:1034] running on GCE node
I20260929 10:34:00.537238 25405 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:34:00.538376 25405 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:34:00.539582 25405 hybrid_clock.cc:648] HybridClock initialized: now 1790678040539552 us; error 43 us; skew 500 ppm
I20260929 10:34:00.541496 25405 webserver.cc:460] Webserver started at http://127.15.82.126:34079/ using document root <none> and password file <none>
I20260929 10:34:00.542079 25405 fs_manager.cc:359] Metadata directory not provided
I20260929 10:34:00.542148 25405 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:34:00.542448 25405 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:34:00.544734 25405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790677981592896-15689-0/minicluster-data/master-0/data/instance:
uuid: "d38f70a3ce7041c1a7139867b5052744"
format_stamp: "Formatted at 2026-09-29 10:34:00 on dist-test-slave-vg89"
server_key: "cd8b0af255a4cf5c3876a751e58ac56e"
server_key_iv: "4abd5c033f97a372c6f560efa1212e3f"
server_key_version: "encryptionkey@0"
I20260929 10:34:00.545394 25405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790677981592896-15689-0/minicluster-data/master-0/wal/instance:
uuid: "d38f70a3ce7041c1a7139867b5052744"
format_stamp: "Formatted at 2026-09-29 10:34:00 on dist-test-slave-vg89"
server_key: "cd8b0af255a4cf5c3876a751e58ac56e"
server_key_iv: "4abd5c033f97a372c6f560efa1212e3f"
server_key_version: "encryptionkey@0"
I20260929 10:34:00.549306 25405 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260929 10:34:00.551831 25419 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:34:00.553064 25405 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260929 10:34:00.553201 25405 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790677981592896-15689-0/minicluster-data/master-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790677981592896-15689-0/minicluster-data/master-0/wal
uuid: "d38f70a3ce7041c1a7139867b5052744"
format_stamp: "Formatted at 2026-09-29 10:34:00 on dist-test-slave-vg89"
server_key: "cd8b0af255a4cf5c3876a751e58ac56e"
server_key_iv: "4abd5c033f97a372c6f560efa1212e3f"
server_key_version: "encryptionkey@0"
I20260929 10:34:00.553316 25405 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790677981592896-15689-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790677981592896-15689-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790677981592896-15689-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:34:00.558008 25405 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:34:00.558679 25405 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:34:00.558862 25405 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:34:00.565517 25405 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.126:41189
I20260929 10:34:00.565538 25471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.126:41189 every 8 connection(s)
I20260929 10:34:00.566506 25405 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
I20260929 10:34:00.567982 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 25405
I20260929 10:34:00.568106 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790677981592896-15689-0/minicluster-data/master-0/wal/instance
I20260929 10:34:00.568389 15689 external_mini_cluster.cc:1428] Setting key e7a120d87f8ee576125c8d7bcfa0ef44
I20260929 10:34:00.569989 25472 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:34:00.573882 25475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:34:00.569696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34402 (local address 127.15.82.126:41189)
0929 10:34:00.570300 (+ 604us) server_negotiation.cc:206] Beginning negotiation
0929 10:34:00.570337 (+ 37us) server_negotiation.cc:399] Waiting for connection header
0929 10:34:00.570374 (+ 37us) server_negotiation.cc:407] Connection header received
0929 10:34:00.571086 (+ 712us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:34:00.571103 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:34:00.571487 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:34:00.571799 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:34:00.572615 (+ 816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:34:00.572620 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:34:00.572631 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:34:00.572641 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:34:00.572694 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:34:00.572710 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:34:00.572813 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0929 10:34:00.572941 (+ 128us) server_negotiation.cc:299] Negotiation successful
0929 10:34:00.573200 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":134,"threads_started":1}
I20260929 10:34:00.576144 25472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d38f70a3ce7041c1a7139867b5052744: Bootstrap starting.
I20260929 10:34:00.578220 25472 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d38f70a3ce7041c1a7139867b5052744: Neither blocks nor log segments found. Creating new log.
I20260929 10:34:00.578994 25472 log.cc:824] T 00000000000000000000000000000000 P d38f70a3ce7041c1a7139867b5052744: Log is configured to *not* fsync() on all Append() calls
I20260929 10:34:00.580868 25472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d38f70a3ce7041c1a7139867b5052744: No bootstrap required, opened a new log
I20260929 10:34:00.584364 25472 raft_consensus.cc:348] T 00000000000000000000000000000000 P d38f70a3ce7041c1a7139867b5052744 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d38f70a3ce7041c1a7139867b5052744" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 41189 } }
I20260929 10:34:00.584622 25472 raft_consensus.cc:374] T 00000000000000000000000000000000 P d38f70a3ce7041c1a7139867b5052744 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:34:00.584695 25472 raft_consensus.cc:729] T 00000000000000000000000000000000 P d38f70a3ce7041c1a7139867b5052744 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d38f70a3ce7041c1a7139867b5052744, State: Initialized, Role: FOLLOWER
I20260929 10:34:00.585143 25472 consensus_queue.cc:260] T 00000000000000000000000000000000 P d38f70a3ce7041c1a7139867b5052744 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d38f70a3ce7041c1a7139867b5052744" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 41189 } }
I20260929 10:34:00.585289 25472 raft_consensus.cc:388] T 00000000000000000000000000000000 P d38f70a3ce7041c1a7139867b5052744 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:34:00.585376 25472 raft_consensus.cc:482] T 00000000000000000000000000000000 P d38f70a3ce7041c1a7139867b5052744 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:34:00.585471 25472 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d38f70a3ce7041c1a7139867b5052744 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:34:00.586369 25472 raft_consensus.cc:504] T 00000000000000000000000000000000 P d38f70a3ce7041c1a7139867b5052744 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d38f70a3ce7041c1a7139867b5052744" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 41189 } }
I20260929 10:34:00.586747 25472 leader_election.cc:302] T 00000000000000000000000000000000 P d38f70a3ce7041c1a7139867b5052744 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d38f70a3ce7041c1a7139867b5052744; no voters:
I20260929 10:34:00.587054 25472 leader_election.cc:288] T 00000000000000000000000000000000 P d38f70a3ce7041c1a7139867b5052744 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:34:00.587185 25477 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d38f70a3ce7041c1a7139867b5052744 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:34:00.587423 25477 raft_consensus.cc:686] T 00000000000000000000000000000000 P d38f70a3ce7041c1a7139867b5052744 [term 1 LEADER]: Becoming Leader. State: Replica: d38f70a3ce7041c1a7139867b5052744, State: Running, Role: LEADER
I20260929 10:34:00.587716 25477 consensus_queue.cc:237] T 00000000000000000000000000000000 P d38f70a3ce7041c1a7139867b5052744 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d38f70a3ce7041c1a7139867b5052744" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 41189 } }
I20260929 10:34:00.588264 25472 sys_catalog.cc:564] T 00000000000000000000000000000000 P d38f70a3ce7041c1a7139867b5052744 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:34:00.589326 25478 sys_catalog.cc:455] T 00000000000000000000000000000000 P d38f70a3ce7041c1a7139867b5052744 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d38f70a3ce7041c1a7139867b5052744" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d38f70a3ce7041c1a7139867b5052744" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 41189 } } }
I20260929 10:34:00.589449 25478 sys_catalog.cc:458] T 00000000000000000000000000000000 P d38f70a3ce7041c1a7139867b5052744 [sys.catalog]: This master's current role is: LEADER
I20260929 10:34:00.589745 25479 sys_catalog.cc:455] T 00000000000000000000000000000000 P d38f70a3ce7041c1a7139867b5052744 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d38f70a3ce7041c1a7139867b5052744. Latest consensus state: current_term: 1 leader_uuid: "d38f70a3ce7041c1a7139867b5052744" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d38f70a3ce7041c1a7139867b5052744" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 41189 } } }
I20260929 10:34:00.589879 25479 sys_catalog.cc:458] T 00000000000000000000000000000000 P d38f70a3ce7041c1a7139867b5052744 [sys.catalog]: This master's current role is: LEADER
I20260929 10:34:00.589952 25486 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:34:00.593134 25486 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:34:00.598644 25486 catalog_manager.cc:1347] Generated new cluster ID: 3f49f5794e504495b81af737bfa36840
I20260929 10:34:00.598721 25486 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:34:00.613839 25486 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:34:00.614167 25486 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:34:00.623828 25486 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d38f70a3ce7041c1a7139867b5052744: Generated new TSK 0
I20260929 10:34:00.624676 25486 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 10:34:00.635421 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.65:0
--local_ip_for_outbound_sockets=127.15.82.65
--webserver_interface=127.15.82.65
--webserver_port=0
--tserver_master_addrs=127.15.82.126:41189
--builtin_ntp_servers=127.15.82.84:42755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260929 10:34:00.748929 25496 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:34:00.749212 25496 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:34:00.749251 25496 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:34:00.753353 25496 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:34:00.753408 25496 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:34:00.753463 25496 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.65
I20260929 10:34:00.758291 25496 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:42755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.82.65
--webserver_port=0
--tserver_master_addrs=127.15.82.126:41189
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25496
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.65
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:34:00.759455 25496 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:34:00.760854 25496 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:34:00.770285 25504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:34:00.770290 25501 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:34:00.770326 25502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:34:00.770542 25496 server_base.cc:1034] running on GCE node
I20260929 10:34:00.770987 25496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:34:00.771646 25496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:34:00.772823 25496 hybrid_clock.cc:648] HybridClock initialized: now 1790678040772808 us; error 36 us; skew 500 ppm
I20260929 10:34:00.774696 25496 webserver.cc:460] Webserver started at http://127.15.82.65:35167/ using document root <none> and password file <none>
I20260929 10:34:00.775259 25496 fs_manager.cc:359] Metadata directory not provided
I20260929 10:34:00.775354 25496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:34:00.775585 25496 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:34:00.777628 25496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-0/data/instance:
uuid: "93d6b9e2456d4aa2ba4f0838c134a3b7"
format_stamp: "Formatted at 2026-09-29 10:34:00 on dist-test-slave-vg89"
server_key: "f5872ea521f30208dd771adb3f6b1fea"
server_key_iv: "a7cbe5eb2f8cbb77f4250e89a863a5b7"
server_key_version: "encryptionkey@0"
I20260929 10:34:00.778182 25496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-0/wal/instance:
uuid: "93d6b9e2456d4aa2ba4f0838c134a3b7"
format_stamp: "Formatted at 2026-09-29 10:34:00 on dist-test-slave-vg89"
server_key: "f5872ea521f30208dd771adb3f6b1fea"
server_key_iv: "a7cbe5eb2f8cbb77f4250e89a863a5b7"
server_key_version: "encryptionkey@0"
I20260929 10:34:00.782094 25496 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260929 10:34:00.784595 25510 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:34:00.785739 25496 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 10:34:00.785866 25496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-0/wal
uuid: "93d6b9e2456d4aa2ba4f0838c134a3b7"
format_stamp: "Formatted at 2026-09-29 10:34:00 on dist-test-slave-vg89"
server_key: "f5872ea521f30208dd771adb3f6b1fea"
server_key_iv: "a7cbe5eb2f8cbb77f4250e89a863a5b7"
server_key_version: "encryptionkey@0"
I20260929 10:34:00.785987 25496 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:34:00.790653 25496 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:34:00.791221 25496 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:34:00.791394 25496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:34:00.791924 25496 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:34:00.792779 25496 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:34:00.792845 25496 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:34:00.792908 25496 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:34:00.792953 25496 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:34:00.801232 25496 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.65:39775
I20260929 10:34:00.801254 25623 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.65:39775 every 8 connection(s)
I20260929 10:34:00.802086 25496 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
I20260929 10:34:00.807801 25475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:34:00.803876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.65:52159 (local address 127.15.82.126:41189)
0929 10:34:00.804017 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0929 10:34:00.804020 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:34:00.805050 (+ 1030us) server_negotiation.cc:407] Connection header received
0929 10:34:00.806293 (+ 1243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:34:00.806295 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:34:00.806334 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:34:00.806454 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:34:00.807428 (+ 974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:34:00.807431 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:34:00.807433 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:34:00.807444 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:34:00.807463 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:34:00.807471 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:34:00.807546 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0929 10:34:00.807658 (+ 112us) server_negotiation.cc:299] Negotiation successful
0929 10:34:00.807694 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260929 10:34:00.808621 25626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:34:00.804274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:41189 (local address 127.15.82.65:52159)
0929 10:34:00.804898 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0929 10:34:00.804944 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0929 10:34:00.806113 (+ 1169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:34:00.806587 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:34:00.806593 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:34:00.806680 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:34:00.807173 (+ 493us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 10:34:00.807175 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:34:00.807213 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 10:34:00.807279 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:34:00.807543 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:34:00.807549 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:34:00.807553 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 10:34:00.807678 (+ 125us) client_negotiation.cc:240] Negotiation successful
0929 10:34:00.807881 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":102,"threads_started":1}
I20260929 10:34:00.809794 25624 heartbeater.cc:347] Connected to a master server at 127.15.82.126:41189
I20260929 10:34:00.809994 25624 heartbeater.cc:464] Registering TS with master...
I20260929 10:34:00.810379 25624 heartbeater.cc:511] Master 127.15.82.126:41189 requested a full tablet report, sending...
I20260929 10:34:00.811833 25436 ts_manager.cc:194] Registered new tserver with Master: 93d6b9e2456d4aa2ba4f0838c134a3b7 (127.15.82.65:39775)
I20260929 10:34:00.812104 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 25496
I20260929 10:34:00.812222 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-0/wal/instance
I20260929 10:34:00.812512 15689 external_mini_cluster.cc:1428] Setting key dfad048f0bd92822f75d30f1154135c0
I20260929 10:34:00.814370 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.66:0
--local_ip_for_outbound_sockets=127.15.82.66
--webserver_interface=127.15.82.66
--webserver_port=0
--tserver_master_addrs=127.15.82.126:41189
--builtin_ntp_servers=127.15.82.84:42755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260929 10:34:00.924338 25627 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:34:00.924593 25627 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:34:00.924650 25627 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:34:00.928663 25627 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:34:00.928718 25627 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:34:00.928773 25627 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.66
I20260929 10:34:00.933513 25627 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:42755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.82.66
--webserver_port=0
--tserver_master_addrs=127.15.82.126:41189
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25627
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.66
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:34:00.934616 25627 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:34:00.935963 25627 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:34:00.945399 25633 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:34:00.945449 25632 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:34:00.945426 25627 server_base.cc:1034] running on GCE node
W20260929 10:34:00.945551 25635 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:34:00.945941 25627 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:34:00.946622 25627 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:34:00.947821 25627 hybrid_clock.cc:648] HybridClock initialized: now 1790678040947800 us; error 34 us; skew 500 ppm
I20260929 10:34:00.949515 25627 webserver.cc:460] Webserver started at http://127.15.82.66:37603/ using document root <none> and password file <none>
I20260929 10:34:00.950089 25627 fs_manager.cc:359] Metadata directory not provided
I20260929 10:34:00.950176 25627 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:34:00.950446 25627 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:34:00.952457 25627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-1/data/instance:
uuid: "76e24884a92f4f9eb6a3ec406ba55bad"
format_stamp: "Formatted at 2026-09-29 10:34:00 on dist-test-slave-vg89"
server_key: "7a429b3f69970c5d65c495f3ac2dd9ab"
server_key_iv: "9a1cacd866b2680a95771252e4c6a7e9"
server_key_version: "encryptionkey@0"
I20260929 10:34:00.953011 25627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-1/wal/instance:
uuid: "76e24884a92f4f9eb6a3ec406ba55bad"
format_stamp: "Formatted at 2026-09-29 10:34:00 on dist-test-slave-vg89"
server_key: "7a429b3f69970c5d65c495f3ac2dd9ab"
server_key_iv: "9a1cacd866b2680a95771252e4c6a7e9"
server_key_version: "encryptionkey@0"
I20260929 10:34:00.956617 25627 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260929 10:34:00.959108 25641 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:34:00.960304 25627 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260929 10:34:00.960424 25627 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-1/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-1/wal
uuid: "76e24884a92f4f9eb6a3ec406ba55bad"
format_stamp: "Formatted at 2026-09-29 10:34:00 on dist-test-slave-vg89"
server_key: "7a429b3f69970c5d65c495f3ac2dd9ab"
server_key_iv: "9a1cacd866b2680a95771252e4c6a7e9"
server_key_version: "encryptionkey@0"
I20260929 10:34:00.960528 25627 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:34:00.964936 25627 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:34:00.965507 25627 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:34:00.965675 25627 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:34:00.966185 25627 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:34:00.967135 25627 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:34:00.967200 25627 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:34:00.967265 25627 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:34:00.967311 25627 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:34:00.975391 25627 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.66:40859
I20260929 10:34:00.975410 25754 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.66:40859 every 8 connection(s)
I20260929 10:34:00.976281 25627 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
I20260929 10:34:00.980657 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 25627
I20260929 10:34:00.980798 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-1/wal/instance
I20260929 10:34:00.981110 15689 external_mini_cluster.cc:1428] Setting key 5068b11543bd26774feebfd98607f381
I20260929 10:34:00.982527 25475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:34:00.978053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:53411 (local address 127.15.82.126:41189)
0929 10:34:00.978190 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0929 10:34:00.978194 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:34:00.979246 (+ 1052us) server_negotiation.cc:407] Connection header received
0929 10:34:00.980583 (+ 1337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:34:00.980586 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:34:00.980623 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:34:00.980697 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:34:00.982074 (+ 1377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:34:00.982078 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:34:00.982080 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:34:00.982091 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:34:00.982113 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:34:00.982121 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:34:00.982206 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0929 10:34:00.982326 (+ 120us) server_negotiation.cc:299] Negotiation successful
0929 10:34:00.982368 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 10:34:00.983270 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.67:0
--local_ip_for_outbound_sockets=127.15.82.67
--webserver_interface=127.15.82.67
--webserver_port=0
--tserver_master_addrs=127.15.82.126:41189
--builtin_ntp_servers=127.15.82.84:42755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260929 10:34:00.983419 25757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:34:00.978489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:41189 (local address 127.15.82.66:53411)
0929 10:34:00.979122 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0929 10:34:00.979162 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0929 10:34:00.980341 (+ 1179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:34:00.980899 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:34:00.980907 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:34:00.981030 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:34:00.981787 (+ 757us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 10:34:00.981789 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:34:00.981853 (+ 64us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 10:34:00.981938 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:34:00.982203 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:34:00.982210 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:34:00.982215 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 10:34:00.982340 (+ 125us) client_negotiation.cc:240] Negotiation successful
0929 10:34:00.982630 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":100,"threads_started":1}
I20260929 10:34:00.984555 25755 heartbeater.cc:347] Connected to a master server at 127.15.82.126:41189
I20260929 10:34:00.984774 25755 heartbeater.cc:464] Registering TS with master...
I20260929 10:34:00.985165 25755 heartbeater.cc:511] Master 127.15.82.126:41189 requested a full tablet report, sending...
I20260929 10:34:00.986120 25436 ts_manager.cc:194] Registered new tserver with Master: 76e24884a92f4f9eb6a3ec406ba55bad (127.15.82.66:40859)
W20260929 10:34:01.095211 25758 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:34:01.095458 25758 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:34:01.095531 25758 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:34:01.099519 25758 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:34:01.099567 25758 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:34:01.099659 25758 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.67
I20260929 10:34:01.104790 25758 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:42755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.82.67
--webserver_port=0
--tserver_master_addrs=127.15.82.126:41189
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25758
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.67
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:34:01.105786 25758 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:34:01.107138 25758 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:34:01.116253 25763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:34:01.116284 25766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:34:01.116231 25764 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:34:01.116381 25758 server_base.cc:1034] running on GCE node
I20260929 10:34:01.116853 25758 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:34:01.117481 25758 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:34:01.118630 25758 hybrid_clock.cc:648] HybridClock initialized: now 1790678041118614 us; error 28 us; skew 500 ppm
I20260929 10:34:01.120486 25758 webserver.cc:460] Webserver started at http://127.15.82.67:34365/ using document root <none> and password file <none>
I20260929 10:34:01.121044 25758 fs_manager.cc:359] Metadata directory not provided
I20260929 10:34:01.121104 25758 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:34:01.121343 25758 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:34:01.123382 25758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-2/data/instance:
uuid: "53b52347977a40f98c2d301c5928b891"
format_stamp: "Formatted at 2026-09-29 10:34:01 on dist-test-slave-vg89"
server_key: "0061ca6cad5764a5f7d1fb1241e62546"
server_key_iv: "51e8a43ea0de9651cf3d5a04e6e7cb80"
server_key_version: "encryptionkey@0"
I20260929 10:34:01.123945 25758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-2/wal/instance:
uuid: "53b52347977a40f98c2d301c5928b891"
format_stamp: "Formatted at 2026-09-29 10:34:01 on dist-test-slave-vg89"
server_key: "0061ca6cad5764a5f7d1fb1241e62546"
server_key_iv: "51e8a43ea0de9651cf3d5a04e6e7cb80"
server_key_version: "encryptionkey@0"
I20260929 10:34:01.127525 25758 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260929 10:34:01.130100 25772 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:34:01.131340 25758 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260929 10:34:01.131470 25758 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-2/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-2/wal
uuid: "53b52347977a40f98c2d301c5928b891"
format_stamp: "Formatted at 2026-09-29 10:34:01 on dist-test-slave-vg89"
server_key: "0061ca6cad5764a5f7d1fb1241e62546"
server_key_iv: "51e8a43ea0de9651cf3d5a04e6e7cb80"
server_key_version: "encryptionkey@0"
I20260929 10:34:01.131579 25758 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:34:01.136096 25758 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:34:01.136678 25758 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:34:01.136845 25758 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:34:01.137346 25758 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:34:01.138196 25758 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:34:01.138258 25758 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:34:01.138320 25758 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:34:01.138367 25758 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:34:01.147384 25758 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.67:37505
I20260929 10:34:01.147399 25885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.67:37505 every 8 connection(s)
I20260929 10:34:01.148270 25758 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
I20260929 10:34:01.150050 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 25758
I20260929 10:34:01.150157 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790677981592896-15689-0/minicluster-data/ts-2/wal/instance
I20260929 10:34:01.150480 15689 external_mini_cluster.cc:1428] Setting key 2a4be046877d4e8fddfbd1386bcc0f6c
I20260929 10:34:01.154035 25475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:34:01.150068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.67:43195 (local address 127.15.82.126:41189)
0929 10:34:01.150231 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0929 10:34:01.150236 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:34:01.151404 (+ 1168us) server_negotiation.cc:407] Connection header received
0929 10:34:01.152671 (+ 1267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:34:01.152674 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:34:01.152710 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:34:01.152774 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:34:01.153716 (+ 942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:34:01.153719 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:34:01.153721 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:34:01.153731 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:34:01.153750 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:34:01.153758 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:34:01.153826 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0929 10:34:01.153900 (+ 74us) server_negotiation.cc:299] Negotiation successful
0929 10:34:01.153933 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260929 10:34:01.154963 25888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:34:01.150528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:41189 (local address 127.15.82.67:43195)
0929 10:34:01.151266 (+ 738us) negotiation.cc:107] Waiting for socket to connect
0929 10:34:01.151317 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0929 10:34:01.152496 (+ 1179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:34:01.152902 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:34:01.152908 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:34:01.152993 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:34:01.153488 (+ 495us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 10:34:01.153489 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:34:01.153527 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 10:34:01.153591 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:34:01.153812 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:34:01.153816 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:34:01.153819 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 10:34:01.153935 (+ 116us) client_negotiation.cc:240] Negotiation successful
0929 10:34:01.154169 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":114,"threads_started":1}
I20260929 10:34:01.155987 25886 heartbeater.cc:347] Connected to a master server at 127.15.82.126:41189
I20260929 10:34:01.156206 25886 heartbeater.cc:464] Registering TS with master...
I20260929 10:34:01.156636 25886 heartbeater.cc:511] Master 127.15.82.126:41189 requested a full tablet report, sending...
I20260929 10:34:01.157629 25436 ts_manager.cc:194] Registered new tserver with Master: 53b52347977a40f98c2d301c5928b891 (127.15.82.67:37505)
I20260929 10:34:01.164075 15689 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260929 10:34:01.166833 25475 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:34:01.166011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34410 (local address 127.15.82.126:41189)
0929 10:34:01.166124 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0929 10:34:01.166127 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:34:01.166287 (+ 160us) server_negotiation.cc:407] Connection header received
0929 10:34:01.166386 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:34:01.166388 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:34:01.166454 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:34:01.166524 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:34:01.166699 (+ 175us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34410: BlockingRecv error: recv got EOF from 127.0.0.1:34410 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
I20260929 10:34:01.167167 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 25496
I20260929 10:34:01.172909 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 25627
I20260929 10:34:01.178154 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 25758
I20260929 10:34:01.183326 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 25405
2026-09-29T10:34:01Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (810 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:34:01 dist-test-slave-vg89 krb5kdc[25896](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:34:01 dist-test-slave-vg89 krb5kdc[25896](info): set up 2 sockets
Sep 29 10:34:01 dist-test-slave-vg89 krb5kdc[25896](info): commencing operation
krb5kdc: starting...
W20260929 10:34:02.231710 15689 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Sep 29 10:34:02 dist-test-slave-vg89 krb5kdc[25896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678042, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:34:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:34:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.82.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab.
Entry for principal kudu/127.15.82.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab.
I20260929 10:34:02.358177 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:45381
--webserver_interface=127.15.82.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab
--principal=kudu/127.15.82.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:37501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.82.126:45381
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:34:02.469980 25909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:34:02.470263 25909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:34:02.470352 25909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:34:02.474529 25909 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:34:02.474588 25909 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:34:02.474608 25909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:34:02.474655 25909 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:34:02.474684 25909 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:34:02.480149 25909 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:37501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.82.126:45381
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab
--principal=kudu/127.15.82.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:45381
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.82.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25909
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:34:02.481240 25909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:34:02.482300 25909 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:34:02.491070 25917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:34:02.491098 25909 server_base.cc:1034] running on GCE node
W20260929 10:34:02.491067 25914 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:34:02.491106 25915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:34:02.491865 25909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:34:02.492962 25909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:34:02.494133 25909 hybrid_clock.cc:648] HybridClock initialized: now 1790678042494103 us; error 46 us; skew 500 ppm
Sep 29 10:34:02 dist-test-slave-vg89 krb5kdc[25896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678042, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:34:02.505662 25909 init.cc:377] Logged in from keytab as kudu/127.15.82.126@KRBTEST.COM (short username kudu)
I20260929 10:34:02.506721 25909 webserver.cc:460] Webserver started at http://127.15.82.126:41689/ using document root <none> and password file <none>
I20260929 10:34:02.507310 25909 fs_manager.cc:359] Metadata directory not provided
I20260929 10:34:02.507400 25909 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:34:02.507632 25909 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:34:02.509716 25909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/minicluster-data/master-0/data/instance:
uuid: "15b6feb6fc5f4073a89617645568f486"
format_stamp: "Formatted at 2026-09-29 10:34:02 on dist-test-slave-vg89"
server_key: "b80dd8b29183d983f2f3e666bb8ef299"
server_key_iv: "f6c290cf85c64b40b0e900dcb08a2137"
server_key_version: "encryptionkey@0"
I20260929 10:34:02.510284 25909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/minicluster-data/master-0/wal/instance:
uuid: "15b6feb6fc5f4073a89617645568f486"
format_stamp: "Formatted at 2026-09-29 10:34:02 on dist-test-slave-vg89"
server_key: "b80dd8b29183d983f2f3e666bb8ef299"
server_key_iv: "f6c290cf85c64b40b0e900dcb08a2137"
server_key_version: "encryptionkey@0"
I20260929 10:34:02.513903 25909 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.002s
I20260929 10:34:02.516395 25924 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:34:02.517617 25909 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260929 10:34:02.517759 25909 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/minicluster-data/master-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/minicluster-data/master-0/wal
uuid: "15b6feb6fc5f4073a89617645568f486"
format_stamp: "Formatted at 2026-09-29 10:34:02 on dist-test-slave-vg89"
server_key: "b80dd8b29183d983f2f3e666bb8ef299"
server_key_iv: "f6c290cf85c64b40b0e900dcb08a2137"
server_key_version: "encryptionkey@0"
I20260929 10:34:02.517864 25909 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:34:02.536916 25909 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:34:02.540329 25909 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:34:02.540547 25909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:34:02.547587 25909 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.126:45381
I20260929 10:34:02.547936 25976 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.126:45381 every 8 connection(s)
I20260929 10:34:02.548684 25909 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
I20260929 10:34:02.551512 25977 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:34:02.554956 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 25909
I20260929 10:34:02.555052 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/minicluster-data/master-0/wal/instance
I20260929 10:34:02.555348 15689 external_mini_cluster.cc:1428] Setting key 9227f298bba9f3a9d8d9cc4c91a4d8b3
I20260929 10:34:02.557641 25977 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15b6feb6fc5f4073a89617645568f486: Bootstrap starting.
I20260929 10:34:02.560359 25977 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 15b6feb6fc5f4073a89617645568f486: Neither blocks nor log segments found. Creating new log.
I20260929 10:34:02.561317 25977 log.cc:824] T 00000000000000000000000000000000 P 15b6feb6fc5f4073a89617645568f486: Log is configured to *not* fsync() on all Append() calls
I20260929 10:34:02.563604 25977 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15b6feb6fc5f4073a89617645568f486: No bootstrap required, opened a new log
Sep 29 10:34:02 dist-test-slave-vg89 krb5kdc[25896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678042, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:34:02.566812 25977 raft_consensus.cc:348] T 00000000000000000000000000000000 P 15b6feb6fc5f4073a89617645568f486 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15b6feb6fc5f4073a89617645568f486" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 45381 } }
I20260929 10:34:02.567035 25977 raft_consensus.cc:374] T 00000000000000000000000000000000 P 15b6feb6fc5f4073a89617645568f486 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:34:02.567121 25977 raft_consensus.cc:729] T 00000000000000000000000000000000 P 15b6feb6fc5f4073a89617645568f486 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15b6feb6fc5f4073a89617645568f486, State: Initialized, Role: FOLLOWER
I20260929 10:34:02.567549 25977 consensus_queue.cc:260] T 00000000000000000000000000000000 P 15b6feb6fc5f4073a89617645568f486 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15b6feb6fc5f4073a89617645568f486" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 45381 } }
I20260929 10:34:02.567711 25977 raft_consensus.cc:388] T 00000000000000000000000000000000 P 15b6feb6fc5f4073a89617645568f486 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:34:02.567781 25977 raft_consensus.cc:482] T 00000000000000000000000000000000 P 15b6feb6fc5f4073a89617645568f486 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:34:02.567914 25977 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 15b6feb6fc5f4073a89617645568f486 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:34:02.568817 25977 raft_consensus.cc:504] T 00000000000000000000000000000000 P 15b6feb6fc5f4073a89617645568f486 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15b6feb6fc5f4073a89617645568f486" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 45381 } }
I20260929 10:34:02.569167 25977 leader_election.cc:302] T 00000000000000000000000000000000 P 15b6feb6fc5f4073a89617645568f486 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 15b6feb6fc5f4073a89617645568f486; no voters:
I20260929 10:34:02.569497 25977 leader_election.cc:288] T 00000000000000000000000000000000 P 15b6feb6fc5f4073a89617645568f486 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:34:02.569633 25982 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 15b6feb6fc5f4073a89617645568f486 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:34:02.569836 25982 raft_consensus.cc:686] T 00000000000000000000000000000000 P 15b6feb6fc5f4073a89617645568f486 [term 1 LEADER]: Becoming Leader. State: Replica: 15b6feb6fc5f4073a89617645568f486, State: Running, Role: LEADER
I20260929 10:34:02.570152 25982 consensus_queue.cc:237] T 00000000000000000000000000000000 P 15b6feb6fc5f4073a89617645568f486 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15b6feb6fc5f4073a89617645568f486" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 45381 } }
I20260929 10:34:02.570897 25977 sys_catalog.cc:564] T 00000000000000000000000000000000 P 15b6feb6fc5f4073a89617645568f486 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:34:02.570935 25980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:34:02.556674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35218 (local address 127.15.82.126:45381)
0929 10:34:02.557066 (+ 392us) server_negotiation.cc:206] Beginning negotiation
0929 10:34:02.557081 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 10:34:02.557116 (+ 35us) server_negotiation.cc:407] Connection header received
0929 10:34:02.557827 (+ 711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:34:02.557852 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:34:02.558182 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:34:02.558506 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:34:02.559478 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:34:02.560493 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:34:02.561690 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:34:02.562312 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:34:02.562484 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:34:02.565223 (+ 2739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:34:02.565228 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:34:02.565241 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:34:02.565266 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:34:02.567690 (+ 2424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:34:02.568192 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:34:02.568198 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:34:02.568203 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:34:02.568323 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:34:02.568603 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:34:02.568606 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:34:02.568608 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:34:02.569043 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:34:02.569237 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0929 10:34:02.569519 (+ 282us) server_negotiation.cc:299] Negotiation successful
0929 10:34:02.570012 (+ 493us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":132,"threads_started":1}
I20260929 10:34:02.571867 25983 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15b6feb6fc5f4073a89617645568f486 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "15b6feb6fc5f4073a89617645568f486" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15b6feb6fc5f4073a89617645568f486" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 45381 } } }
I20260929 10:34:02.572023 25983 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15b6feb6fc5f4073a89617645568f486 [sys.catalog]: This master's current role is: LEADER
I20260929 10:34:02.572387 25984 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15b6feb6fc5f4073a89617645568f486 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 15b6feb6fc5f4073a89617645568f486. Latest consensus state: current_term: 1 leader_uuid: "15b6feb6fc5f4073a89617645568f486" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15b6feb6fc5f4073a89617645568f486" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 45381 } } }
I20260929 10:34:02.572505 25984 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15b6feb6fc5f4073a89617645568f486 [sys.catalog]: This master's current role is: LEADER
I20260929 10:34:02.573505 25990 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:34:02.576385 25990 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:34:02.581779 25990 catalog_manager.cc:1347] Generated new cluster ID: 14fafdbf0f2b40228715e2a1d03e7c36
I20260929 10:34:02.581846 25990 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:34:02.593918 25990 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:34:02.594889 25990 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:34:02.602344 25990 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 15b6feb6fc5f4073a89617645568f486: Generated new TSK 0
I20260929 10:34:02.603116 25990 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.82.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab.
Entry for principal kudu/127.15.82.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab.
I20260929 10:34:02.641741 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.65:0
--local_ip_for_outbound_sockets=127.15.82.65
--webserver_interface=127.15.82.65
--webserver_port=0
--tserver_master_addrs=127.15.82.126:45381
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab
--principal=kudu/127.15.82.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:37501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:34:02.752993 26003 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:34:02.753265 26003 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:34:02.753309 26003 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:34:02.757432 26003 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:34:02.757493 26003 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:34:02.757591 26003 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.65
I20260929 10:34:02.763007 26003 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:37501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab
--principal=kudu/127.15.82.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.82.65
--webserver_port=0
--tserver_master_addrs=127.15.82.126:45381
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26003
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.65
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:34:02.764106 26003 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:34:02.765206 26003 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:34:02.775063 26008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:34:02.775127 26009 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:34:02.775137 26011 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:34:02.775337 26003 server_base.cc:1034] running on GCE node
I20260929 10:34:02.775727 26003 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:34:02.776337 26003 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:34:02.777489 26003 hybrid_clock.cc:648] HybridClock initialized: now 1790678042777450 us; error 48 us; skew 500 ppm
Sep 29 10:34:02 dist-test-slave-vg89 krb5kdc[25896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678042, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:34:02.780449 26003 init.cc:377] Logged in from keytab as kudu/127.15.82.65@KRBTEST.COM (short username kudu)
I20260929 10:34:02.781327 26003 webserver.cc:460] Webserver started at http://127.15.82.65:41197/ using document root <none> and password file <none>
I20260929 10:34:02.781884 26003 fs_manager.cc:359] Metadata directory not provided
I20260929 10:34:02.781947 26003 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:34:02.782159 26003 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:34:02.784193 26003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/minicluster-data/ts-0/data/instance:
uuid: "fcf5a577f2e3403d9fe8303012227e6a"
format_stamp: "Formatted at 2026-09-29 10:34:02 on dist-test-slave-vg89"
server_key: "e5106e5c5d67521412a0e107544f8cbc"
server_key_iv: "791818e5a101913df163693ac7f2e43f"
server_key_version: "encryptionkey@0"
I20260929 10:34:02.784736 26003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/minicluster-data/ts-0/wal/instance:
uuid: "fcf5a577f2e3403d9fe8303012227e6a"
format_stamp: "Formatted at 2026-09-29 10:34:02 on dist-test-slave-vg89"
server_key: "e5106e5c5d67521412a0e107544f8cbc"
server_key_iv: "791818e5a101913df163693ac7f2e43f"
server_key_version: "encryptionkey@0"
I20260929 10:34:02.788517 26003 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 10:34:02.790999 26018 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:34:02.792261 26003 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 10:34:02.792371 26003 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/minicluster-data/ts-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/minicluster-data/ts-0/wal
uuid: "fcf5a577f2e3403d9fe8303012227e6a"
format_stamp: "Formatted at 2026-09-29 10:34:02 on dist-test-slave-vg89"
server_key: "e5106e5c5d67521412a0e107544f8cbc"
server_key_iv: "791818e5a101913df163693ac7f2e43f"
server_key_version: "encryptionkey@0"
I20260929 10:34:02.792481 26003 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:34:02.811563 26003 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:34:02.814338 26003 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:34:02.814576 26003 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:34:02.815160 26003 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:34:02.815943 26003 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:34:02.815987 26003 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:34:02.816051 26003 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:34:02.816082 26003 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:34:02.825057 26003 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.65:35813
I20260929 10:34:02.825083 26131 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.65:35813 every 8 connection(s)
I20260929 10:34:02.825977 26003 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
I20260929 10:34:02.828406 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 26003
I20260929 10:34:02.828541 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/minicluster-data/ts-0/wal/instance
I20260929 10:34:02.828863 15689 external_mini_cluster.cc:1428] Setting key cf3a4476774d783e388acb2d7e65a696
Sep 29 10:34:02 dist-test-slave-vg89 krb5kdc[25896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678042, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.65@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.82.66@KRBTEST.COM; defaulting to no policy
I20260929 10:34:02.841043 25980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:34:02.827715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.65:51493 (local address 127.15.82.126:45381)
0929 10:34:02.827845 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0929 10:34:02.827849 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:34:02.828775 (+ 926us) server_negotiation.cc:407] Connection header received
0929 10:34:02.830328 (+ 1553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:34:02.830331 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:34:02.830380 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:34:02.830475 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:34:02.831615 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:34:02.832369 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:34:02.834239 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:34:02.834510 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:34:02.834590 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:34:02.837513 (+ 2923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:34:02.837516 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:34:02.837518 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:34:02.837539 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:34:02.839021 (+ 1482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:34:02.839528 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:34:02.839533 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:34:02.839538 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:34:02.839625 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:34:02.840004 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:34:02.840009 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:34:02.840010 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:34:02.840184 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:34:02.840290 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0929 10:34:02.840762 (+ 472us) server_negotiation.cc:299] Negotiation successful
0929 10:34:02.840883 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260929 10:34:02.841866 26134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:34:02.828119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:45381 (local address 127.15.82.65:51493)
0929 10:34:02.828616 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0929 10:34:02.828668 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0929 10:34:02.830075 (+ 1407us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:34:02.830640 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:34:02.830647 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:34:02.830996 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:34:02.831474 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:34:02.831487 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:34:02.832496 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:34:02.832501 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:34:02.834099 (+ 1598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:34:02.834125 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:34:02.834664 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:34:02.834672 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:34:02.834924 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:34:02.835622 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:34:02.835646 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:34:02.837348 (+ 1702us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:34:02.839134 (+ 1786us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:34:02.839140 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:34:02.839156 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:34:02.839426 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:34:02.839719 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:34:02.839722 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:34:02.839724 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:34:02.839914 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:34:02.840292 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:34:02.840301 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:34:02.840544 (+ 243us) client_negotiation.cc:769] Sending connection context
0929 10:34:02.840724 (+ 180us) client_negotiation.cc:240] Negotiation successful
0929 10:34:02.841051 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":139,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.66@KRBTEST.COM" created.
I20260929 10:34:02.843127 26132 heartbeater.cc:347] Connected to a master server at 127.15.82.126:45381
I20260929 10:34:02.843390 26132 heartbeater.cc:464] Registering TS with master...
I20260929 10:34:02.843977 26132 heartbeater.cc:511] Master 127.15.82.126:45381 requested a full tablet report, sending...
I20260929 10:34:02.845522 25941 ts_manager.cc:194] Registered new tserver with Master: fcf5a577f2e3403d9fe8303012227e6a (127.15.82.65:35813)
I20260929 10:34:02.846758 25941 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.82.65@KRBTEST.COM'} at 127.15.82.65:51493
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab.
Entry for principal kudu/127.15.82.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab.
I20260929 10:34:02.857228 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.66:0
--local_ip_for_outbound_sockets=127.15.82.66
--webserver_interface=127.15.82.66
--webserver_port=0
--tserver_master_addrs=127.15.82.126:45381
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab
--principal=kudu/127.15.82.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:37501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:34:02.964799 26137 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:34:02.965090 26137 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:34:02.965171 26137 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:34:02.969178 26137 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:34:02.969265 26137 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:34:02.969357 26137 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.66
I20260929 10:34:02.974784 26137 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:37501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab
--principal=kudu/127.15.82.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.82.66
--webserver_port=0
--tserver_master_addrs=127.15.82.126:45381
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26137
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.66
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:34:02.975854 26137 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:34:02.976869 26137 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:34:02.986068 26142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:34:02.986127 26137 server_base.cc:1034] running on GCE node
W20260929 10:34:02.986081 26143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:34:02.986068 26145 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:34:02.986649 26137 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:34:02.987291 26137 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:34:02.988448 26137 hybrid_clock.cc:648] HybridClock initialized: now 1790678042988418 us; error 42 us; skew 500 ppm
Sep 29 10:34:02 dist-test-slave-vg89 krb5kdc[25896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678042, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:34:02.991055 26137 init.cc:377] Logged in from keytab as kudu/127.15.82.66@KRBTEST.COM (short username kudu)
I20260929 10:34:02.991859 26137 webserver.cc:460] Webserver started at http://127.15.82.66:44417/ using document root <none> and password file <none>
I20260929 10:34:02.992403 26137 fs_manager.cc:359] Metadata directory not provided
I20260929 10:34:02.992487 26137 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:34:02.992726 26137 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:34:02.994766 26137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/minicluster-data/ts-1/data/instance:
uuid: "59ead66013cc47038fbf0db9d001f36d"
format_stamp: "Formatted at 2026-09-29 10:34:02 on dist-test-slave-vg89"
server_key: "534d2a5c74fcd06ffe82fd658bc191ea"
server_key_iv: "3bf297c6bfa949813492397c69c13493"
server_key_version: "encryptionkey@0"
I20260929 10:34:02.995347 26137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/minicluster-data/ts-1/wal/instance:
uuid: "59ead66013cc47038fbf0db9d001f36d"
format_stamp: "Formatted at 2026-09-29 10:34:02 on dist-test-slave-vg89"
server_key: "534d2a5c74fcd06ffe82fd658bc191ea"
server_key_iv: "3bf297c6bfa949813492397c69c13493"
server_key_version: "encryptionkey@0"
I20260929 10:34:02.998904 26137 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260929 10:34:03.001534 26152 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:34:03.002691 26137 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 10:34:03.002784 26137 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/minicluster-data/ts-1/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/minicluster-data/ts-1/wal
uuid: "59ead66013cc47038fbf0db9d001f36d"
format_stamp: "Formatted at 2026-09-29 10:34:02 on dist-test-slave-vg89"
server_key: "534d2a5c74fcd06ffe82fd658bc191ea"
server_key_iv: "3bf297c6bfa949813492397c69c13493"
server_key_version: "encryptionkey@0"
I20260929 10:34:03.002907 26137 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:34:03.012179 26137 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:34:03.014873 26137 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:34:03.015055 26137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:34:03.015594 26137 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:34:03.016367 26137 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:34:03.016412 26137 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:34:03.016472 26137 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:34:03.016510 26137 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:34:03.025465 26137 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.66:42225
I20260929 10:34:03.025488 26266 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.66:42225 every 8 connection(s)
I20260929 10:34:03.026331 26137 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
I20260929 10:34:03.033560 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 26137
I20260929 10:34:03.033659 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/minicluster-data/ts-1/wal/instance
I20260929 10:34:03.033950 15689 external_mini_cluster.cc:1428] Setting key 796700765ed6fa45d4a8d74fa1ebbbc0
Sep 29 10:34:03 dist-test-slave-vg89 krb5kdc[25896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678042, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.66@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:34:03.041635 25980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:34:03.028422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:50561 (local address 127.15.82.126:45381)
0929 10:34:03.028549 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0929 10:34:03.028552 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:34:03.029373 (+ 821us) server_negotiation.cc:407] Connection header received
0929 10:34:03.030946 (+ 1573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:34:03.030950 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:34:03.031014 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:34:03.031104 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:34:03.032374 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:34:03.032975 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:34:03.034391 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:34:03.034787 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:34:03.034904 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:34:03.037660 (+ 2756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:34:03.037670 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:34:03.037672 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:34:03.037697 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:34:03.039453 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:34:03.040018 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:34:03.040023 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:34:03.040025 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:34:03.040114 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:34:03.040485 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:34:03.040491 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:34:03.040493 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:34:03.040697 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:34:03.040809 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0929 10:34:03.041321 (+ 512us) server_negotiation.cc:299] Negotiation successful
0929 10:34:03.041452 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260929 10:34:03.042608 26269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:34:03.028779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:45381 (local address 127.15.82.66:50561)
0929 10:34:03.029221 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0929 10:34:03.029261 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0929 10:34:03.030592 (+ 1331us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:34:03.031289 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:34:03.031297 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:34:03.031682 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:34:03.032200 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:34:03.032214 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:34:03.033112 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:34:03.033116 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:34:03.034254 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:34:03.034265 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:34:03.034955 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:34:03.034959 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:34:03.035190 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:34:03.035866 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:34:03.035884 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:34:03.037483 (+ 1599us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:34:03.039601 (+ 2118us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:34:03.039607 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:34:03.039623 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:34:03.039906 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:34:03.040224 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:34:03.040229 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:34:03.040232 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:34:03.040383 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:34:03.040807 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:34:03.040814 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:34:03.041074 (+ 260us) client_negotiation.cc:769] Sending connection context
0929 10:34:03.041274 (+ 200us) client_negotiation.cc:240] Negotiation successful
0929 10:34:03.041616 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":116,"threads_started":1}
WARNING: no policy specified for kudu/127.15.82.67@KRBTEST.COM; defaulting to no policy
I20260929 10:34:03.043861 26267 heartbeater.cc:347] Connected to a master server at 127.15.82.126:45381
I20260929 10:34:03.044075 26267 heartbeater.cc:464] Registering TS with master...
I20260929 10:34:03.044548 26267 heartbeater.cc:511] Master 127.15.82.126:45381 requested a full tablet report, sending...
I20260929 10:34:03.045555 25941 ts_manager.cc:194] Registered new tserver with Master: 59ead66013cc47038fbf0db9d001f36d (127.15.82.66:42225)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.67@KRBTEST.COM" created.
I20260929 10:34:03.046391 25941 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.82.66@KRBTEST.COM'} at 127.15.82.66:50561
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab.
Entry for principal kudu/127.15.82.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab.
I20260929 10:34:03.061998 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.67:0
--local_ip_for_outbound_sockets=127.15.82.67
--webserver_interface=127.15.82.67
--webserver_port=0
--tserver_master_addrs=127.15.82.126:45381
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab
--principal=kudu/127.15.82.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:37501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:34:03.174074 26272 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:34:03.174312 26272 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:34:03.174389 26272 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:34:03.178352 26272 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:34:03.178424 26272 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:34:03.178530 26272 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.67
I20260929 10:34:03.183627 26272 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:37501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab
--principal=kudu/127.15.82.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.82.67
--webserver_port=0
--tserver_master_addrs=127.15.82.126:45381
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26272
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.67
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:34:03.184595 26272 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:34:03.185576 26272 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:34:03.194772 26280 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:34:03.194828 26272 server_base.cc:1034] running on GCE node
W20260929 10:34:03.194789 26278 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:34:03.194777 26277 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:34:03.195354 26272 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:34:03.196038 26272 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:34:03.197255 26272 hybrid_clock.cc:648] HybridClock initialized: now 1790678043197234 us; error 34 us; skew 500 ppm
Sep 29 10:34:03 dist-test-slave-vg89 krb5kdc[25896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678043, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:34:03.199934 26272 init.cc:377] Logged in from keytab as kudu/127.15.82.67@KRBTEST.COM (short username kudu)
I20260929 10:34:03.200795 26272 webserver.cc:460] Webserver started at http://127.15.82.67:33997/ using document root <none> and password file <none>
I20260929 10:34:03.201336 26272 fs_manager.cc:359] Metadata directory not provided
I20260929 10:34:03.201421 26272 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:34:03.201642 26272 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:34:03.203660 26272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/minicluster-data/ts-2/data/instance:
uuid: "6e39a517a7534659a32337b72b1d2b4c"
format_stamp: "Formatted at 2026-09-29 10:34:03 on dist-test-slave-vg89"
server_key: "602a509fcafe2e4e4973018ea5a45cdd"
server_key_iv: "ebd1f571908003aec999a8f18b1bd4c0"
server_key_version: "encryptionkey@0"
I20260929 10:34:03.204224 26272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/minicluster-data/ts-2/wal/instance:
uuid: "6e39a517a7534659a32337b72b1d2b4c"
format_stamp: "Formatted at 2026-09-29 10:34:03 on dist-test-slave-vg89"
server_key: "602a509fcafe2e4e4973018ea5a45cdd"
server_key_iv: "ebd1f571908003aec999a8f18b1bd4c0"
server_key_version: "encryptionkey@0"
I20260929 10:34:03.207808 26272 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260929 10:34:03.210129 26287 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:34:03.211315 26272 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 10:34:03.211447 26272 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/minicluster-data/ts-2/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/minicluster-data/ts-2/wal
uuid: "6e39a517a7534659a32337b72b1d2b4c"
format_stamp: "Formatted at 2026-09-29 10:34:03 on dist-test-slave-vg89"
server_key: "602a509fcafe2e4e4973018ea5a45cdd"
server_key_iv: "ebd1f571908003aec999a8f18b1bd4c0"
server_key_version: "encryptionkey@0"
I20260929 10:34:03.211548 26272 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:34:03.230772 26272 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:34:03.233477 26272 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:34:03.233671 26272 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:34:03.234241 26272 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:34:03.235042 26272 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:34:03.235110 26272 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:34:03.235193 26272 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:34:03.235241 26272 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:34:03.244298 26272 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.67:43617
I20260929 10:34:03.244325 26400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.67:43617 every 8 connection(s)
I20260929 10:34:03.245182 26272 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
I20260929 10:34:03.248556 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 26272
I20260929 10:34:03.248704 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/minicluster-data/ts-2/wal/instance
I20260929 10:34:03.249042 15689 external_mini_cluster.cc:1428] Setting key 4a007ab5e0d4046463592ba48f8e76f7
Sep 29 10:34:03 dist-test-slave-vg89 krb5kdc[25896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678043, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.67@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:34:03.258793 25980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:34:03.247016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.67:35969 (local address 127.15.82.126:45381)
0929 10:34:03.247147 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0929 10:34:03.247151 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:34:03.247998 (+ 847us) server_negotiation.cc:407] Connection header received
0929 10:34:03.249728 (+ 1730us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:34:03.249731 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:34:03.249786 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:34:03.249866 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:34:03.251126 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:34:03.251664 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:34:03.252725 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:34:03.252935 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:34:03.253011 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:34:03.255398 (+ 2387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:34:03.255402 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:34:03.255404 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:34:03.255448 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:34:03.256874 (+ 1426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:34:03.257350 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:34:03.257353 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:34:03.257354 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:34:03.257448 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:34:03.257741 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:34:03.257744 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:34:03.257745 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:34:03.257935 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:34:03.258019 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0929 10:34:03.258535 (+ 516us) server_negotiation.cc:299] Negotiation successful
0929 10:34:03.258665 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260929 10:34:03.259547 26403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:34:03.247418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:45381 (local address 127.15.82.67:35969)
0929 10:34:03.247864 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0929 10:34:03.247905 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 10:34:03.249534 (+ 1629us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:34:03.250037 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:34:03.250046 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:34:03.250490 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:34:03.250969 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:34:03.250982 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:34:03.251772 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:34:03.251774 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:34:03.252623 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:34:03.252631 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:34:03.253055 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:34:03.253057 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:34:03.253192 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:34:03.253775 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:34:03.253791 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:34:03.255230 (+ 1439us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:34:03.256975 (+ 1745us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:34:03.256980 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:34:03.256991 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:34:03.257235 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:34:03.257535 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:34:03.257537 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:34:03.257538 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:34:03.257661 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:34:03.258021 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:34:03.258026 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:34:03.258242 (+ 216us) client_negotiation.cc:769] Sending connection context
0929 10:34:03.258426 (+ 184us) client_negotiation.cc:240] Negotiation successful
0929 10:34:03.258687 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":116,"threads_started":1}
I20260929 10:34:03.260707 26401 heartbeater.cc:347] Connected to a master server at 127.15.82.126:45381
I20260929 10:34:03.260941 26401 heartbeater.cc:464] Registering TS with master...
I20260929 10:34:03.261474 26401 heartbeater.cc:511] Master 127.15.82.126:45381 requested a full tablet report, sending...
I20260929 10:34:03.262549 25941 ts_manager.cc:194] Registered new tserver with Master: 6e39a517a7534659a32337b72b1d2b4c (127.15.82.67:43617)
I20260929 10:34:03.263115 25941 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.82.67@KRBTEST.COM'} at 127.15.82.67:35969
I20260929 10:34:03.274726 15689 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 10:34:03.283587 25980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:34:03.276659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35234 (local address 127.15.82.126:45381)
0929 10:34:03.276801 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0929 10:34:03.276804 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:34:03.276947 (+ 143us) server_negotiation.cc:407] Connection header received
0929 10:34:03.277044 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:34:03.277047 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:34:03.277091 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:34:03.277165 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:34:03.277764 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:34:03.278293 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:34:03.279122 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:34:03.279332 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:34:03.279425 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:34:03.280732 (+ 1307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:34:03.280735 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:34:03.280737 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:34:03.280757 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:34:03.282001 (+ 1244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:34:03.282355 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:34:03.282357 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:34:03.282359 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:34:03.282468 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:34:03.282687 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:34:03.282690 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:34:03.282691 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:34:03.282893 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:34:03.283023 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0929 10:34:03.283268 (+ 245us) server_negotiation.cc:299] Negotiation successful
0929 10:34:03.283402 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260929 10:34:03.286353 25941 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35234:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 10:34:03.289359 25941 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:34:03.302551 26412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:34:03.297439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.65:35813 (local address 127.0.0.1:46192)
0929 10:34:03.297802 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0929 10:34:03.297858 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0929 10:34:03.297987 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:34:03.298602 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:34:03.298607 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:34:03.298627 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:34:03.298779 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:34:03.298787 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:34:03.299826 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:34:03.299833 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:34:03.301059 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:34:03.301067 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:34:03.302028 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:34:03.302032 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:34:03.302138 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:34:03.302180 (+ 42us) client_negotiation.cc:769] Sending connection context
0929 10:34:03.302263 (+ 83us) client_negotiation.cc:240] Negotiation successful
0929 10:34:03.302378 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":92,"threads_started":1}
I20260929 10:34:03.303411 26413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:34:03.297593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46192 (local address 127.15.82.65:35813)
0929 10:34:03.297942 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0929 10:34:03.297958 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0929 10:34:03.297984 (+ 26us) server_negotiation.cc:407] Connection header received
0929 10:34:03.298194 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:34:03.298202 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:34:03.298469 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:34:03.298618 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:34:03.298913 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:34:03.299695 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:34:03.301187 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:34:03.301900 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:34:03.302919 (+ 1019us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:34:03.303084 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0929 10:34:03.303141 (+ 57us) server_negotiation.cc:299] Negotiation successful
0929 10:34:03.303246 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":67,"threads_started":1}
I20260929 10:34:03.303609 26414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:34:03.297837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.66:42225 (local address 127.0.0.1:56040)
0929 10:34:03.298353 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0929 10:34:03.298366 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 10:34:03.298469 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:34:03.299704 (+ 1235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:34:03.299707 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:34:03.299724 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:34:03.299822 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:34:03.299828 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:34:03.301190 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:34:03.301194 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:34:03.302204 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:34:03.302213 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:34:03.303193 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:34:03.303196 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:34:03.303285 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:34:03.303308 (+ 23us) client_negotiation.cc:769] Sending connection context
0929 10:34:03.303396 (+ 88us) client_negotiation.cc:240] Negotiation successful
0929 10:34:03.303479 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":106,"threads_started":1}
I20260929 10:34:03.303681 26416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:34:03.298623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56040 (local address 127.15.82.66:42225)
0929 10:34:03.299340 (+ 717us) server_negotiation.cc:206] Beginning negotiation
0929 10:34:03.299348 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 10:34:03.299376 (+ 28us) server_negotiation.cc:407] Connection header received
0929 10:34:03.299437 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:34:03.299440 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:34:03.299577 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:34:03.299729 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:34:03.299914 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:34:03.301043 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:34:03.302324 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:34:03.303067 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:34:03.303192 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:34:03.303328 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0929 10:34:03.303422 (+ 94us) server_negotiation.cc:299] Negotiation successful
0929 10:34:03.303515 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":632,"thread_start_us":103,"threads_started":1}
I20260929 10:34:03.305042 26415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:34:03.298199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.67:43617 (local address 127.0.0.1:54416)
0929 10:34:03.299954 (+ 1755us) negotiation.cc:107] Waiting for socket to connect
0929 10:34:03.299967 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 10:34:03.300058 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:34:03.300487 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:34:03.300490 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:34:03.300507 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:34:03.300579 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:34:03.300584 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:34:03.301899 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:34:03.301903 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:34:03.303013 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:34:03.303023 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:34:03.304682 (+ 1659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:34:03.304685 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:34:03.304770 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:34:03.304791 (+ 21us) client_negotiation.cc:769] Sending connection context
0929 10:34:03.304859 (+ 68us) client_negotiation.cc:240] Negotiation successful
0929 10:34:03.304921 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1695,"thread_start_us":86,"threads_started":1}
I20260929 10:34:03.305454 26417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:34:03.298890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54416 (local address 127.15.82.67:43617)
0929 10:34:03.299206 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0929 10:34:03.299219 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 10:34:03.300063 (+ 844us) server_negotiation.cc:407] Connection header received
0929 10:34:03.300175 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:34:03.300179 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:34:03.300352 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:34:03.300489 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:34:03.300669 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:34:03.301764 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:34:03.303134 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:34:03.304391 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:34:03.304496 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:34:03.304618 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0929 10:34:03.305219 (+ 601us) server_negotiation.cc:299] Negotiation successful
0929 10:34:03.305315 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":124,"threads_started":1}
I20260929 10:34:03.306346 26201 tablet_service.cc:1467] Processing CreateTablet for tablet 87b70ae25e6a40a899c80d4d180058ac (DEFAULT_TABLE table=test-table [id=7e7a32f133a64abe828dae1c9878a485]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:34:03.306354 26066 tablet_service.cc:1467] Processing CreateTablet for tablet 87b70ae25e6a40a899c80d4d180058ac (DEFAULT_TABLE table=test-table [id=7e7a32f133a64abe828dae1c9878a485]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:34:03.307348 26201 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87b70ae25e6a40a899c80d4d180058ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:34:03.307348 26066 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87b70ae25e6a40a899c80d4d180058ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:34:03.313098 26335 tablet_service.cc:1467] Processing CreateTablet for tablet 87b70ae25e6a40a899c80d4d180058ac (DEFAULT_TABLE table=test-table [id=7e7a32f133a64abe828dae1c9878a485]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:34:03.314014 26419 tablet_bootstrap.cc:492] T 87b70ae25e6a40a899c80d4d180058ac P fcf5a577f2e3403d9fe8303012227e6a: Bootstrap starting.
I20260929 10:34:03.314167 26335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87b70ae25e6a40a899c80d4d180058ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:34:03.315100 26418 tablet_bootstrap.cc:492] T 87b70ae25e6a40a899c80d4d180058ac P 59ead66013cc47038fbf0db9d001f36d: Bootstrap starting.
I20260929 10:34:03.316491 26419 tablet_bootstrap.cc:654] T 87b70ae25e6a40a899c80d4d180058ac P fcf5a577f2e3403d9fe8303012227e6a: Neither blocks nor log segments found. Creating new log.
I20260929 10:34:03.317802 26418 tablet_bootstrap.cc:654] T 87b70ae25e6a40a899c80d4d180058ac P 59ead66013cc47038fbf0db9d001f36d: Neither blocks nor log segments found. Creating new log.
I20260929 10:34:03.317858 26419 log.cc:824] T 87b70ae25e6a40a899c80d4d180058ac P fcf5a577f2e3403d9fe8303012227e6a: Log is configured to *not* fsync() on all Append() calls
I20260929 10:34:03.318661 26418 log.cc:824] T 87b70ae25e6a40a899c80d4d180058ac P 59ead66013cc47038fbf0db9d001f36d: Log is configured to *not* fsync() on all Append() calls
I20260929 10:34:03.320053 26419 tablet_bootstrap.cc:492] T 87b70ae25e6a40a899c80d4d180058ac P fcf5a577f2e3403d9fe8303012227e6a: No bootstrap required, opened a new log
I20260929 10:34:03.320253 26419 ts_tablet_manager.cc:1397] T 87b70ae25e6a40a899c80d4d180058ac P fcf5a577f2e3403d9fe8303012227e6a: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260929 10:34:03.320281 26422 tablet_bootstrap.cc:492] T 87b70ae25e6a40a899c80d4d180058ac P 6e39a517a7534659a32337b72b1d2b4c: Bootstrap starting.
I20260929 10:34:03.320498 26418 tablet_bootstrap.cc:492] T 87b70ae25e6a40a899c80d4d180058ac P 59ead66013cc47038fbf0db9d001f36d: No bootstrap required, opened a new log
I20260929 10:34:03.320685 26418 ts_tablet_manager.cc:1397] T 87b70ae25e6a40a899c80d4d180058ac P 59ead66013cc47038fbf0db9d001f36d: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20260929 10:34:03.322295 26422 tablet_bootstrap.cc:654] T 87b70ae25e6a40a899c80d4d180058ac P 6e39a517a7534659a32337b72b1d2b4c: Neither blocks nor log segments found. Creating new log.
I20260929 10:34:03.323300 26422 log.cc:824] T 87b70ae25e6a40a899c80d4d180058ac P 6e39a517a7534659a32337b72b1d2b4c: Log is configured to *not* fsync() on all Append() calls
I20260929 10:34:03.324247 26419 raft_consensus.cc:348] T 87b70ae25e6a40a899c80d4d180058ac P fcf5a577f2e3403d9fe8303012227e6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf5a577f2e3403d9fe8303012227e6a" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 35813 } } peers { permanent_uuid: "6e39a517a7534659a32337b72b1d2b4c" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 43617 } } peers { permanent_uuid: "59ead66013cc47038fbf0db9d001f36d" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 42225 } }
I20260929 10:34:03.324522 26419 raft_consensus.cc:374] T 87b70ae25e6a40a899c80d4d180058ac P fcf5a577f2e3403d9fe8303012227e6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:34:03.324414 26418 raft_consensus.cc:348] T 87b70ae25e6a40a899c80d4d180058ac P 59ead66013cc47038fbf0db9d001f36d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf5a577f2e3403d9fe8303012227e6a" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 35813 } } peers { permanent_uuid: "6e39a517a7534659a32337b72b1d2b4c" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 43617 } } peers { permanent_uuid: "59ead66013cc47038fbf0db9d001f36d" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 42225 } }
I20260929 10:34:03.324574 26419 raft_consensus.cc:729] T 87b70ae25e6a40a899c80d4d180058ac P fcf5a577f2e3403d9fe8303012227e6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf5a577f2e3403d9fe8303012227e6a, State: Initialized, Role: FOLLOWER
I20260929 10:34:03.324612 26418 raft_consensus.cc:374] T 87b70ae25e6a40a899c80d4d180058ac P 59ead66013cc47038fbf0db9d001f36d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:34:03.324671 26418 raft_consensus.cc:729] T 87b70ae25e6a40a899c80d4d180058ac P 59ead66013cc47038fbf0db9d001f36d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59ead66013cc47038fbf0db9d001f36d, State: Initialized, Role: FOLLOWER
I20260929 10:34:03.324975 26422 tablet_bootstrap.cc:492] T 87b70ae25e6a40a899c80d4d180058ac P 6e39a517a7534659a32337b72b1d2b4c: No bootstrap required, opened a new log
I20260929 10:34:03.325033 26418 consensus_queue.cc:260] T 87b70ae25e6a40a899c80d4d180058ac P 59ead66013cc47038fbf0db9d001f36d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf5a577f2e3403d9fe8303012227e6a" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 35813 } } peers { permanent_uuid: "6e39a517a7534659a32337b72b1d2b4c" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 43617 } } peers { permanent_uuid: "59ead66013cc47038fbf0db9d001f36d" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 42225 } }
I20260929 10:34:03.325033 26419 consensus_queue.cc:260] T 87b70ae25e6a40a899c80d4d180058ac P fcf5a577f2e3403d9fe8303012227e6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf5a577f2e3403d9fe8303012227e6a" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 35813 } } peers { permanent_uuid: "6e39a517a7534659a32337b72b1d2b4c" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 43617 } } peers { permanent_uuid: "59ead66013cc47038fbf0db9d001f36d" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 42225 } }
I20260929 10:34:03.325171 26422 ts_tablet_manager.cc:1397] T 87b70ae25e6a40a899c80d4d180058ac P 6e39a517a7534659a32337b72b1d2b4c: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260929 10:34:03.325687 26267 heartbeater.cc:503] Master 127.15.82.126:45381 was elected leader, sending a full tablet report...
I20260929 10:34:03.326112 26418 ts_tablet_manager.cc:1428] T 87b70ae25e6a40a899c80d4d180058ac P 59ead66013cc47038fbf0db9d001f36d: Time spent starting tablet: real 0.005s user 0.004s sys 0.002s
I20260929 10:34:03.326882 26132 heartbeater.cc:503] Master 127.15.82.126:45381 was elected leader, sending a full tablet report...
W20260929 10:34:03.327029 26133 tablet.cc:2369] T 87b70ae25e6a40a899c80d4d180058ac P fcf5a577f2e3403d9fe8303012227e6a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 10:34:03.327438 26419 ts_tablet_manager.cc:1428] T 87b70ae25e6a40a899c80d4d180058ac P fcf5a577f2e3403d9fe8303012227e6a: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260929 10:34:03.328899 26422 raft_consensus.cc:348] T 87b70ae25e6a40a899c80d4d180058ac P 6e39a517a7534659a32337b72b1d2b4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf5a577f2e3403d9fe8303012227e6a" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 35813 } } peers { permanent_uuid: "6e39a517a7534659a32337b72b1d2b4c" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 43617 } } peers { permanent_uuid: "59ead66013cc47038fbf0db9d001f36d" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 42225 } }
I20260929 10:34:03.329144 26422 raft_consensus.cc:374] T 87b70ae25e6a40a899c80d4d180058ac P 6e39a517a7534659a32337b72b1d2b4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:34:03.329202 26422 raft_consensus.cc:729] T 87b70ae25e6a40a899c80d4d180058ac P 6e39a517a7534659a32337b72b1d2b4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e39a517a7534659a32337b72b1d2b4c, State: Initialized, Role: FOLLOWER
I20260929 10:34:03.329612 26422 consensus_queue.cc:260] T 87b70ae25e6a40a899c80d4d180058ac P 6e39a517a7534659a32337b72b1d2b4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf5a577f2e3403d9fe8303012227e6a" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 35813 } } peers { permanent_uuid: "6e39a517a7534659a32337b72b1d2b4c" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 43617 } } peers { permanent_uuid: "59ead66013cc47038fbf0db9d001f36d" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 42225 } }
I20260929 10:34:03.330259 26401 heartbeater.cc:503] Master 127.15.82.126:45381 was elected leader, sending a full tablet report...
I20260929 10:34:03.330624 26422 ts_tablet_manager.cc:1428] T 87b70ae25e6a40a899c80d4d180058ac P 6e39a517a7534659a32337b72b1d2b4c: Time spent starting tablet: real 0.005s user 0.007s sys 0.000s
W20260929 10:34:03.496161 26402 tablet.cc:2369] T 87b70ae25e6a40a899c80d4d180058ac P 6e39a517a7534659a32337b72b1d2b4c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 10:34:03.527599 26268 tablet.cc:2369] T 87b70ae25e6a40a899c80d4d180058ac P 59ead66013cc47038fbf0db9d001f36d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 10:34:03.606614 26424 raft_consensus.cc:482] T 87b70ae25e6a40a899c80d4d180058ac P 59ead66013cc47038fbf0db9d001f36d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:34:03.606822 26424 raft_consensus.cc:504] T 87b70ae25e6a40a899c80d4d180058ac P 59ead66013cc47038fbf0db9d001f36d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf5a577f2e3403d9fe8303012227e6a" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 35813 } } peers { permanent_uuid: "6e39a517a7534659a32337b72b1d2b4c" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 43617 } } peers { permanent_uuid: "59ead66013cc47038fbf0db9d001f36d" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 42225 } }
I20260929 10:34:03.607983 26424 leader_election.cc:288] T 87b70ae25e6a40a899c80d4d180058ac P 59ead66013cc47038fbf0db9d001f36d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fcf5a577f2e3403d9fe8303012227e6a (127.15.82.65:35813), 6e39a517a7534659a32337b72b1d2b4c (127.15.82.67:43617)
I20260929 10:34:03.613660 26413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:34:03.608372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:56373 (local address 127.15.82.65:35813)
0929 10:34:03.608528 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0929 10:34:03.608534 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 10:34:03.608645 (+ 111us) server_negotiation.cc:407] Connection header received
0929 10:34:03.608973 (+ 328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:34:03.608979 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:34:03.609037 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:34:03.609120 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:34:03.609474 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:34:03.610268 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:34:03.612243 (+ 1975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:34:03.612948 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:34:03.613108 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:34:03.613166 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0929 10:34:03.613435 (+ 269us) server_negotiation.cc:299] Negotiation successful
0929 10:34:03.613516 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 10:34:03.613685 26417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:34:03.608769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:42653 (local address 127.15.82.67:43617)
0929 10:34:03.608894 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0929 10:34:03.608898 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:34:03.608990 (+ 92us) server_negotiation.cc:407] Connection header received
0929 10:34:03.609081 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:34:03.609086 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:34:03.609137 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:34:03.609205 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:34:03.609489 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:34:03.610266 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:34:03.612244 (+ 1978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:34:03.612972 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:34:03.613106 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:34:03.613166 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0929 10:34:03.613435 (+ 269us) server_negotiation.cc:299] Negotiation successful
0929 10:34:03.613513 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260929 10:34:03.613708 26428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:34:03.608638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.67:43617 (local address 127.15.82.66:42653)
0929 10:34:03.608903 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0929 10:34:03.608914 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:34:03.609003 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:34:03.609231 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:34:03.609234 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:34:03.609245 (+ 11us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:34:03.609385 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:34:03.609394 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:34:03.610383 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:34:03.610386 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:34:03.612072 (+ 1686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:34:03.612086 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:34:03.613100 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:34:03.613104 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:34:03.613278 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:34:03.613323 (+ 45us) client_negotiation.cc:769] Sending connection context
0929 10:34:03.613397 (+ 74us) client_negotiation.cc:240] Negotiation successful
0929 10:34:03.613515 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"mutex_wait_us":252,"thread_start_us":60,"threads_started":1}
I20260929 10:34:03.613708 26427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:34:03.608232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.65:35813 (local address 127.15.82.66:56373)
0929 10:34:03.608528 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0929 10:34:03.608551 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0929 10:34:03.608736 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:34:03.609144 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:34:03.609146 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:34:03.609157 (+ 11us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:34:03.609335 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:34:03.609347 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:34:03.610383 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:34:03.610386 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:34:03.612072 (+ 1686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:34:03.612086 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:34:03.613100 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:34:03.613104 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:34:03.613274 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:34:03.613319 (+ 45us) client_negotiation.cc:769] Sending connection context
0929 10:34:03.613398 (+ 79us) client_negotiation.cc:240] Negotiation successful
0929 10:34:03.613511 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"mutex_wait_us":260,"thread_start_us":96,"threads_started":1}
I20260929 10:34:03.614673 26355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87b70ae25e6a40a899c80d4d180058ac" candidate_uuid: "59ead66013cc47038fbf0db9d001f36d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e39a517a7534659a32337b72b1d2b4c" is_pre_election: true
I20260929 10:34:03.614696 26086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87b70ae25e6a40a899c80d4d180058ac" candidate_uuid: "59ead66013cc47038fbf0db9d001f36d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf5a577f2e3403d9fe8303012227e6a" is_pre_election: true
I20260929 10:34:03.615031 26355 raft_consensus.cc:2454] T 87b70ae25e6a40a899c80d4d180058ac P 6e39a517a7534659a32337b72b1d2b4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 59ead66013cc47038fbf0db9d001f36d in term 0.
I20260929 10:34:03.615031 26086 raft_consensus.cc:2454] T 87b70ae25e6a40a899c80d4d180058ac P fcf5a577f2e3403d9fe8303012227e6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 59ead66013cc47038fbf0db9d001f36d in term 0.
I20260929 10:34:03.615502 26155 leader_election.cc:302] T 87b70ae25e6a40a899c80d4d180058ac P 59ead66013cc47038fbf0db9d001f36d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 59ead66013cc47038fbf0db9d001f36d, fcf5a577f2e3403d9fe8303012227e6a; no voters:
I20260929 10:34:03.615829 26424 raft_consensus.cc:2781] T 87b70ae25e6a40a899c80d4d180058ac P 59ead66013cc47038fbf0db9d001f36d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:34:03.615940 26424 raft_consensus.cc:482] T 87b70ae25e6a40a899c80d4d180058ac P 59ead66013cc47038fbf0db9d001f36d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:34:03.616017 26424 raft_consensus.cc:3037] T 87b70ae25e6a40a899c80d4d180058ac P 59ead66013cc47038fbf0db9d001f36d [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:34:03.617121 26424 raft_consensus.cc:504] T 87b70ae25e6a40a899c80d4d180058ac P 59ead66013cc47038fbf0db9d001f36d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf5a577f2e3403d9fe8303012227e6a" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 35813 } } peers { permanent_uuid: "6e39a517a7534659a32337b72b1d2b4c" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 43617 } } peers { permanent_uuid: "59ead66013cc47038fbf0db9d001f36d" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 42225 } }
I20260929 10:34:03.617478 26424 leader_election.cc:288] T 87b70ae25e6a40a899c80d4d180058ac P 59ead66013cc47038fbf0db9d001f36d [CANDIDATE]: Term 1 election: Requested vote from peers fcf5a577f2e3403d9fe8303012227e6a (127.15.82.65:35813), 6e39a517a7534659a32337b72b1d2b4c (127.15.82.67:43617)
I20260929 10:34:03.617857 26086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87b70ae25e6a40a899c80d4d180058ac" candidate_uuid: "59ead66013cc47038fbf0db9d001f36d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf5a577f2e3403d9fe8303012227e6a"
I20260929 10:34:03.617900 26355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87b70ae25e6a40a899c80d4d180058ac" candidate_uuid: "59ead66013cc47038fbf0db9d001f36d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e39a517a7534659a32337b72b1d2b4c"
I20260929 10:34:03.617985 26086 raft_consensus.cc:3037] T 87b70ae25e6a40a899c80d4d180058ac P fcf5a577f2e3403d9fe8303012227e6a [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:34:03.618027 26355 raft_consensus.cc:3037] T 87b70ae25e6a40a899c80d4d180058ac P 6e39a517a7534659a32337b72b1d2b4c [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:34:03.619040 26086 raft_consensus.cc:2454] T 87b70ae25e6a40a899c80d4d180058ac P fcf5a577f2e3403d9fe8303012227e6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59ead66013cc47038fbf0db9d001f36d in term 1.
I20260929 10:34:03.619155 26355 raft_consensus.cc:2454] T 87b70ae25e6a40a899c80d4d180058ac P 6e39a517a7534659a32337b72b1d2b4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59ead66013cc47038fbf0db9d001f36d in term 1.
I20260929 10:34:03.619405 26155 leader_election.cc:302] T 87b70ae25e6a40a899c80d4d180058ac P 59ead66013cc47038fbf0db9d001f36d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 59ead66013cc47038fbf0db9d001f36d, fcf5a577f2e3403d9fe8303012227e6a; no voters:
I20260929 10:34:03.619565 26424 raft_consensus.cc:2781] T 87b70ae25e6a40a899c80d4d180058ac P 59ead66013cc47038fbf0db9d001f36d [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:34:03.619791 26424 raft_consensus.cc:686] T 87b70ae25e6a40a899c80d4d180058ac P 59ead66013cc47038fbf0db9d001f36d [term 1 LEADER]: Becoming Leader. State: Replica: 59ead66013cc47038fbf0db9d001f36d, State: Running, Role: LEADER
I20260929 10:34:03.620146 26424 consensus_queue.cc:237] T 87b70ae25e6a40a899c80d4d180058ac P 59ead66013cc47038fbf0db9d001f36d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf5a577f2e3403d9fe8303012227e6a" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 35813 } } peers { permanent_uuid: "6e39a517a7534659a32337b72b1d2b4c" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 43617 } } peers { permanent_uuid: "59ead66013cc47038fbf0db9d001f36d" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 42225 } }
I20260929 10:34:03.623539 25941 catalog_manager.cc:5495] T 87b70ae25e6a40a899c80d4d180058ac P 59ead66013cc47038fbf0db9d001f36d reported cstate change: term changed from 0 to 1, leader changed from <none> to 59ead66013cc47038fbf0db9d001f36d (127.15.82.66). New cstate: current_term: 1 leader_uuid: "59ead66013cc47038fbf0db9d001f36d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf5a577f2e3403d9fe8303012227e6a" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 35813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e39a517a7534659a32337b72b1d2b4c" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 43617 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59ead66013cc47038fbf0db9d001f36d" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 42225 } health_report { overall_health: HEALTHY } } }
I20260929 10:34:03.664924 26416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:34:03.661413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43836 (local address 127.15.82.66:42225)
0929 10:34:03.661573 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0929 10:34:03.661577 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:34:03.661594 (+ 17us) server_negotiation.cc:407] Connection header received
0929 10:34:03.661708 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:34:03.661710 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:34:03.661773 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:34:03.661876 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:34:03.662077 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:34:03.662642 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:34:03.663449 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:34:03.663709 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:34:03.663801 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:34:03.664053 (+ 252us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:34:03.664482 (+ 429us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:34:03.664582 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0929 10:34:03.664662 (+ 80us) server_negotiation.cc:299] Negotiation successful
0929 10:34:03.664750 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260929 10:34:03.670698 26355 raft_consensus.cc:1261] T 87b70ae25e6a40a899c80d4d180058ac P 6e39a517a7534659a32337b72b1d2b4c [term 1 FOLLOWER]: Refusing update from remote peer 59ead66013cc47038fbf0db9d001f36d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:34:03.670720 26086 raft_consensus.cc:1261] T 87b70ae25e6a40a899c80d4d180058ac P fcf5a577f2e3403d9fe8303012227e6a [term 1 FOLLOWER]: Refusing update from remote peer 59ead66013cc47038fbf0db9d001f36d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:34:03.671468 26424 consensus_queue.cc:1035] T 87b70ae25e6a40a899c80d4d180058ac P 59ead66013cc47038fbf0db9d001f36d [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e39a517a7534659a32337b72b1d2b4c" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 43617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:34:03.671701 26429 consensus_queue.cc:1035] T 87b70ae25e6a40a899c80d4d180058ac P 59ead66013cc47038fbf0db9d001f36d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf5a577f2e3403d9fe8303012227e6a" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 35813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:34:03.679642 26433 mvcc.cc:205] Tried to move back new op lower bound from 7334617266868031488 to 7334617266671083520. Current Snapshot: MvccSnapshot[applied={T|T < 7334617266868031488}]
I20260929 10:34:03.688372 25939 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35234:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:34:03.688571 25939 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35234:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:34:03.691238 25939 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 15b6feb6fc5f4073a89617645568f486: Sending DeleteTablet for 3 replicas of tablet 87b70ae25e6a40a899c80d4d180058ac
I20260929 10:34:03.692010 26201 tablet_service.cc:1514] Processing DeleteTablet for tablet 87b70ae25e6a40a899c80d4d180058ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:34:03 UTC) from {username='kudu', principal='kudu/127.15.82.126@KRBTEST.COM'} at 127.0.0.1:56040
I20260929 10:34:03.692080 26335 tablet_service.cc:1514] Processing DeleteTablet for tablet 87b70ae25e6a40a899c80d4d180058ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:34:03 UTC) from {username='kudu', principal='kudu/127.15.82.126@KRBTEST.COM'} at 127.0.0.1:54416
I20260929 10:34:03.692615 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 26003
I20260929 10:34:03.692761 26066 tablet_service.cc:1514] Processing DeleteTablet for tablet 87b70ae25e6a40a899c80d4d180058ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:34:03 UTC) from {username='kudu', principal='kudu/127.15.82.126@KRBTEST.COM'} at 127.0.0.1:46192
I20260929 10:34:03.693184 26442 tablet_replica.cc:330] T 87b70ae25e6a40a899c80d4d180058ac P 59ead66013cc47038fbf0db9d001f36d: stopping tablet replica
I20260929 10:34:03.693513 26442 raft_consensus.cc:2229] T 87b70ae25e6a40a899c80d4d180058ac P 59ead66013cc47038fbf0db9d001f36d [term 1 LEADER]: Raft consensus shutting down.
I20260929 10:34:03.693953 26442 raft_consensus.cc:2258] T 87b70ae25e6a40a899c80d4d180058ac P 59ead66013cc47038fbf0db9d001f36d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:34:03.695639 26442 ts_tablet_manager.cc:1905] T 87b70ae25e6a40a899c80d4d180058ac P 59ead66013cc47038fbf0db9d001f36d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:34:03.696355 26443 tablet_replica.cc:330] T 87b70ae25e6a40a899c80d4d180058ac P 6e39a517a7534659a32337b72b1d2b4c: stopping tablet replica
I20260929 10:34:03.696686 26443 raft_consensus.cc:2229] T 87b70ae25e6a40a899c80d4d180058ac P 6e39a517a7534659a32337b72b1d2b4c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:34:03.697011 26443 raft_consensus.cc:2258] T 87b70ae25e6a40a899c80d4d180058ac P 6e39a517a7534659a32337b72b1d2b4c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:34:03.698549 26443 ts_tablet_manager.cc:1905] T 87b70ae25e6a40a899c80d4d180058ac P 6e39a517a7534659a32337b72b1d2b4c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:34:03.699293 26442 ts_tablet_manager.cc:1918] T 87b70ae25e6a40a899c80d4d180058ac P 59ead66013cc47038fbf0db9d001f36d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:34:03.699461 26442 log.cc:1196] T 87b70ae25e6a40a899c80d4d180058ac P 59ead66013cc47038fbf0db9d001f36d: Deleting WAL directory at /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/minicluster-data/ts-1/wal/wals/87b70ae25e6a40a899c80d4d180058ac
I20260929 10:34:03.699852 26442 ts_tablet_manager.cc:1939] T 87b70ae25e6a40a899c80d4d180058ac P 59ead66013cc47038fbf0db9d001f36d: Deleting consensus metadata
W20260929 10:34:03.700593 25927 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.82.65:35813 (error 108)
I20260929 10:34:03.701195 25928 catalog_manager.cc:4841] TS 59ead66013cc47038fbf0db9d001f36d (127.15.82.66:42225): tablet 87b70ae25e6a40a899c80d4d180058ac (table test-table [id=7e7a32f133a64abe828dae1c9878a485]) successfully deleted
I20260929 10:34:03.701435 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 26137
I20260929 10:34:03.701843 26415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:34:03.701527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.65:35813 (local address 127.0.0.1:55906)
0929 10:34:03.701661 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0929 10:34:03.701741 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.65:35813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260929 10:34:03.702159 25927 catalog_manager.cc:4584] TS fcf5a577f2e3403d9fe8303012227e6a (127.15.82.65:35813): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 87b70ae25e6a40a899c80d4d180058ac: Network error: Client connection negotiation failed: client connection to 127.15.82.65:35813: connect: Connection refused (error 111)
I20260929 10:34:03.703624 26443 ts_tablet_manager.cc:1918] T 87b70ae25e6a40a899c80d4d180058ac P 6e39a517a7534659a32337b72b1d2b4c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:34:03.703745 26443 log.cc:1196] T 87b70ae25e6a40a899c80d4d180058ac P 6e39a517a7534659a32337b72b1d2b4c: Deleting WAL directory at /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790677981592896-15689-0/minicluster-data/ts-2/wal/wals/87b70ae25e6a40a899c80d4d180058ac
I20260929 10:34:03.704087 26443 ts_tablet_manager.cc:1939] T 87b70ae25e6a40a899c80d4d180058ac P 6e39a517a7534659a32337b72b1d2b4c: Deleting consensus metadata
I20260929 10:34:03.705186 25927 catalog_manager.cc:4841] TS 6e39a517a7534659a32337b72b1d2b4c (127.15.82.67:43617): tablet 87b70ae25e6a40a899c80d4d180058ac (table test-table [id=7e7a32f133a64abe828dae1c9878a485]) successfully deleted
I20260929 10:34:03.707810 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 26272
I20260929 10:34:03.713810 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 25909
2026-09-29T10:34:03Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2531 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:34:03 dist-test-slave-vg89 krb5kdc[26447](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:34:03 dist-test-slave-vg89 krb5kdc[26447](info): set up 2 sockets
Sep 29 10:34:03 dist-test-slave-vg89 krb5kdc[26447](info): commencing operation
krb5kdc: starting...
W20260929 10:34:04.799736 15689 mini_kdc.cc:121] Time spent starting KDC: real 1.063s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Sep 29 10:34:04 dist-test-slave-vg89 krb5kdc[26447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678044, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:34:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:34:04Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.15.82.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.15.82.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.15.82.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/krb5kdc/rangeradmin_127.15.82.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.15.82.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/krb5kdc/rangeradmin_127.15.82.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.15.82.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.15.82.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.15.82.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/krb5kdc/rangerlookup_127.15.82.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.15.82.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/krb5kdc/rangerlookup_127.15.82.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.15.82.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.15.82.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.15.82.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/krb5kdc/HTTP_127.15.82.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.15.82.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/krb5kdc/HTTP_127.15.82.84@KRBTEST.COM.keytab.
I20260929 10:34:04.970359 15689 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskiKMUyb/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/postgres -l logfile start
2026-09-29 10:34:07.472 UTC [26477] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-29 10:34:07.472 UTC [26477] LOG: listening on IPv4 address "127.15.82.84", port 54153
2026-09-29 10:34:07.476 UTC [26477] LOG: listening on Unix socket "/tmp/.s.PGSQL.54153"
2026-09-29 10:34:07.497 UTC [26480] LOG: database system was shut down at 2026-09-29 10:34:06 UTC
2026-09-29 10:34:07.500 UTC [26477] LOG: database system is ready to accept connections
I20260929 10:34:08.455631 15689 mini_postgres.cc:96] Postgres bound to 54153
2026-09-29 10:34:08.462 UTC [26488] FATAL: database "slave" does not exist
127.15.82.84:54153 - accepting connections
I20260929 10:34:08.463647 15689 mini_ranger.cc:161] Starting Ranger...
I20260929 10:34:08.480474 15689 mini_ranger.cc:84] Created miniranger Postgres user
I20260929 10:34:08.961880 15689 mini_ranger.cc:87] Created ranger Postgres database
I20260929 10:34:08.961973 15689 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/ranger-admin
2026-09-29 10:34:09,253 [I] DB FLAVOR :POSTGRES
2026-09-29 10:34:09,254 [I] --------- Verifying Ranger DB connection ---------
2026-09-29 10:34:09,254 [I] Checking connection..
2026-09-29 10:34:09,254 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiKMUyb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskiKMUyb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.82.84:54153/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-29 10:34:09,486 [I] Checking connection passed.
2026-09-29 10:34:09,486 [I] --------- Verifying version history table ---------
2026-09-29 10:34:09,486 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiKMUyb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskiKMUyb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.82.84:54153/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 10:34:09,719 [I] Table x_db_version_h does not exist in database ranger
2026-09-29 10:34:09,720 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiKMUyb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskiKMUyb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.82.84:54153/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 10:34:09,976 [I] Table x_db_version_h does not exist in database ranger
2026-09-29 10:34:09,976 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-29 10:34:09,977 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiKMUyb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskiKMUyb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.82.84:54153/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskiKMUyb/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-29 10:34:10.215 UTC [26552] WARNING: there is no transaction in progress
2026-09-29 10:34:10,222 [I] create_dbversion_catalog.sql file imported successfully
2026-09-29 10:34:10,222 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiKMUyb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskiKMUyb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.82.84:54153/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 10:34:10,448 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-29 10:34:10,448 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-29 10:34:10,449 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiKMUyb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskiKMUyb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.82.84:54153/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-29 10:34:10,689 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiKMUyb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskiKMUyb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.82.84:54153/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-29 10:34:10,918 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiKMUyb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskiKMUyb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.82.84:54153/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-vg89.c.gcp-upstream.internal','N') ;"
2026-09-29 10:34:11,151 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiKMUyb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskiKMUyb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.82.84:54153/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-29 10:34:11,377 [I] Table x_portal_user does not exist in database ranger
2026-09-29 10:34:11,378 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiKMUyb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskiKMUyb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.82.84:54153/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-29 10:34:11,635 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-29 10:34:11,636 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiKMUyb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskiKMUyb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.82.84:54153/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-29 10:34:11,861 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiKMUyb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskiKMUyb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.82.84:54153/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-29 10:34:12,108 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-29 10:34:12,108 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiKMUyb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskiKMUyb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.82.84:54153/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskiKMUyb/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-29 10:34:13.083 UTC [26660] WARNING: there is no transaction in progress
2026-09-29 10:34:13.099 UTC [26660] WARNING: there is no transaction in progress
2026-09-29 10:34:13.117 UTC [26660] WARNING: there is no transaction in progress
2026-09-29 10:34:13.131 UTC [26660] WARNING: there is no transaction in progress
2026-09-29 10:34:13.147 UTC [26660] WARNING: there is no transaction in progress
2026-09-29 10:34:13.163 UTC [26660] WARNING: there is no transaction in progress
2026-09-29 10:34:13.216 UTC [26660] WARNING: there is no transaction in progress
2026-09-29 10:34:13.223 UTC [26660] WARNING: there is no transaction in progress
2026-09-29 10:34:13.237 UTC [26660] WARNING: there is no transaction in progress
2026-09-29 10:34:13.247 UTC [26660] WARNING: there is no transaction in progress
2026-09-29 10:34:13.257 UTC [26660] WARNING: there is no transaction in progress
2026-09-29 10:34:13.270 UTC [26660] WARNING: there is no transaction in progress
2026-09-29 10:34:13.280 UTC [26660] WARNING: there is no transaction in progress
2026-09-29 10:34:13.286 UTC [26660] WARNING: there is no transaction in progress
2026-09-29 10:34:13.293 UTC [26660] WARNING: there is no transaction in progress
2026-09-29 10:34:13.300 UTC [26660] WARNING: there is no transaction in progress
2026-09-29 10:34:13,942 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-29 10:34:13,942 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiKMUyb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskiKMUyb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.82.84:54153/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-29 10:34:14,191 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-29 10:34:14,192 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiKMUyb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskiKMUyb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.82.84:54153/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-29 10:34:14,418 [I] Table x_portal_user already exists in database 'ranger'
2026-09-29 10:34:14,418 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiKMUyb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskiKMUyb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.82.84:54153/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-29 10:34:14,654 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-29 10:34:14,655 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiKMUyb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskiKMUyb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.82.84:54153/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-29 10:34:14,908 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiKMUyb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskiKMUyb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.82.84:54153/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-29 10:34:15,158 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiKMUyb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskiKMUyb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.82.84:54153/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-vg89.c.gcp-upstream.internal';"
2026-09-29 10:34:15,393 [I] CORE_DB_SCHEMA import status has been updated
2026-09-29 10:34:15,393 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiKMUyb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskiKMUyb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.82.84:54153/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-29 10:34:15,635 [I] DB_PATCHES have already been applied
I20260929 10:34:15.640983 15689 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/ranger-admin:/tmp/dist-test-taskiKMUyb/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskiKMUyb/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskiKMUyb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskiKMUyb/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260929 10:34:15.641050 15689 mini_ranger.cc:193] Using host: 127.15.82.84
I20260929 10:34:15.644948 15689 mini_ranger.cc:227] Ranger admin URL: http://127.15.82.84:33509
Sep 29, 2026 10:34:15 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 29, 2026 10:34:15 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 29, 2026 10:34:16 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/ranger-admin/ews/webapp
Sep 29, 2026 10:34:16 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 29, 2026 10:34:16 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 29, 2026 10:34:16 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/ranger-admin/ews/webapp].
Sep 29, 2026 10:34:16 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260929 10:34:16.794651 15689 mini_ranger.cc:160] Time spent starting Ranger: real 8.331s user 0.000s sys 0.012s
Sep 29, 2026 10:34:37 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 29, 2026 10:34:37 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.XKeyREST
Sep 29, 2026 10:34:37 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 29, 2026 10:34:37 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 29, 2026 10:34:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 29, 2026 10:34:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 29, 2026 10:34:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 29, 2026 10:34:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 29, 2026 10:34:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 29, 2026 10:34:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 29, 2026 10:34:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 29, 2026 10:34:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 29, 2026 10:34:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 29, 2026 10:34:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 29, 2026 10:34:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 29, 2026 10:34:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 29, 2026 10:34:37 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 29, 2026 10:34:38 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260929 10:34:38.745296 15689 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.15.82.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.15.82.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.15.82.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/krb5kdc/rangerkms_127.15.82.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.15.82.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/krb5kdc/rangerkms_127.15.82.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.15.82.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.15.82.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.15.82.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/krb5kdc/HTTP_127.15.82.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.15.82.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/krb5kdc/HTTP_127.15.82.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 29 10:34:38 dist-test-slave-vg89 krb5kdc[26447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678078, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260929 10:34:38.832885 15689 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260929 10:34:38.842484 15689 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260929 10:34:39.753427 15689 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260929 10:34:39.753525 15689 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/ranger-kms
I20260929 10:34:39.753623 15689 mini_ranger_kms.cc:227] Using postgres at 127.15.82.84:54153
2026-09-29 10:34:39,954 [I] DB FLAVOR :POSTGRES
2026-09-29 10:34:39,954 [I] --------- Verifying Ranger DB connection ---------
2026-09-29 10:34:39,954 [I] Checking connection
2026-09-29 10:34:39,954 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiKMUyb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.82.84:54153/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-29 10:34:40,285 [I] connection success
2026-09-29 10:34:40,286 [I] --------- Verifying Ranger DB tables ---------
2026-09-29 10:34:40,286 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-29 10:34:40,286 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiKMUyb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.82.84:54153/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-29 10:34:40,598 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-29 10:34:40,598 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-29 10:34:40,599 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-29 10:34:40,599 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiKMUyb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.82.84:54153/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-29 10:34:40,915 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260929 10:34:41.377128 15689 mini_ranger_kms.cc:310] Created kms service
I20260929 10:34:41.604453 15689 mini_ranger_kms.cc:326] Created kudu user
I20260929 10:34:41.652510 15689 mini_ranger_kms.cc:343] Created rangerkms user
Sep 29 10:34:41 dist-test-slave-vg89 krb5kdc[26447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678078, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.15.82.84@KRBTEST.COM
I20260929 10:34:42.061787 15689 mini_ranger_kms.cc:384] Added ranger policy
I20260929 10:34:42.062093 15689 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/ranger-kms:/tmp/dist-test-taskiKMUyb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskiKMUyb/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskiKMUyb/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260929 10:34:42.062120 15689 mini_ranger_kms.cc:240] Using host: 127.15.82.84
I20260929 10:34:42.066280 15689 mini_ranger_kms.cc:276] Ranger KMS PID: 26840
I20260929 10:34:42.066447 15689 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.15.82.84:51769
Sep 29, 2026 10:34:43 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 29, 2026 10:34:43 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 29, 2026 10:34:43 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 29, 2026 10:34:43 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 29, 2026 10:34:43 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/ranger-kms/ews/webapp].
Sep 29, 2026 10:34:43 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260929 10:34:44.383631 15689 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.551s user 0.004s sys 0.179s
I20260929 10:34:44.383785 15689 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260929 10:34:44.383920 15689 mini_ranger_kms.cc:401] 127.15.82.84:51769/kms/v1/keys
Sep 29 10:34:44 dist-test-slave-vg89 krb5kdc[26447](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790678084, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.82.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 29 10:34:44 dist-test-slave-vg89 krb5kdc[26447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678084, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.82.84@KRBTEST.COM for HTTP/127.15.82.84@KRBTEST.COM
Sep 29 10:34:45 dist-test-slave-vg89 krb5kdc[26447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678084, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.82.84@KRBTEST.COM for HTTP/127.15.82.84@KRBTEST.COM
Sep 29 10:34:47 dist-test-slave-vg89 krb5kdc[26447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678087, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.15.82.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab.
Entry for principal kudu/127.15.82.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab.
I20260929 10:34:47.462117 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:43935
--webserver_interface=127.15.82.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab
--principal=kudu/127.15.82.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:41699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.82.126:43935
--ranger_config_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.82.84:51769/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:34:47.597615 26891 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:34:47.597882 26891 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:34:47.597921 26891 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:34:47.601795 26891 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:34:47.601854 26891 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:34:47.601873 26891 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:34:47.601889 26891 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:34:47.601905 26891 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:34:47.607272 26891 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:41699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/master-0/wal
--ranger_kms_url=127.15.82.84:51769/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.15.82.126:43935
--ranger_config_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab
--principal=kudu/127.15.82.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:43935
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.82.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26891
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:34:47.608306 26891 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:34:47.609268 26891 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:34:47.617959 26899 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:34:47.618021 26891 server_base.cc:1034] running on GCE node
W20260929 10:34:47.617967 26896 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:34:47.617969 26897 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:34:47.618891 26891 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:34:47.619962 26891 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:34:47.621130 26891 hybrid_clock.cc:648] HybridClock initialized: now 1790678087621100 us; error 42 us; skew 500 ppm
Sep 29 10:34:47 dist-test-slave-vg89 krb5kdc[26447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678087, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:34:47.623920 26891 init.cc:377] Logged in from keytab as kudu/127.15.82.126@KRBTEST.COM (short username kudu)
I20260929 10:34:47.624776 26891 webserver.cc:460] Webserver started at http://127.15.82.126:37257/ using document root <none> and password file <none>
I20260929 10:34:47.625319 26891 fs_manager.cc:359] Metadata directory not provided
I20260929 10:34:47.625375 26891 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:34:47.625586 26891 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 10:34:47 dist-test-slave-vg89 krb5kdc[26447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678087, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.126@KRBTEST.COM for HTTP/127.15.82.84@KRBTEST.COM
I20260929 10:34:47.683298 26891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/master-0/data/instance:
uuid: "ce47f2b3f23d45c59e39437a5b3d02d5"
format_stamp: "Formatted at 2026-09-29 10:34:47 on dist-test-slave-vg89"
server_key: "aafc31f071b8f198d60f37f3ad14a550"
server_key_iv: "3d478a3dc8df9e6638ef2e15e05812ab"
server_key_version: "kuduclusterkey@0"
I20260929 10:34:47.683933 26891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/master-0/wal/instance:
uuid: "ce47f2b3f23d45c59e39437a5b3d02d5"
format_stamp: "Formatted at 2026-09-29 10:34:47 on dist-test-slave-vg89"
server_key: "aafc31f071b8f198d60f37f3ad14a550"
server_key_iv: "3d478a3dc8df9e6638ef2e15e05812ab"
server_key_version: "kuduclusterkey@0"
I20260929 10:34:47.687857 26891 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 10:34:47.704375 26906 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:34:47.705832 26891 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 10:34:47.706002 26891 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/master-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/master-0/wal
uuid: "ce47f2b3f23d45c59e39437a5b3d02d5"
format_stamp: "Formatted at 2026-09-29 10:34:47 on dist-test-slave-vg89"
server_key: "aafc31f071b8f198d60f37f3ad14a550"
server_key_iv: "3d478a3dc8df9e6638ef2e15e05812ab"
server_key_version: "kuduclusterkey@0"
I20260929 10:34:47.706137 26891 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:34:47.717224 26891 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:34:47.720134 26891 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:34:47.720328 26891 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:34:47.727886 26891 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.126:43935
I20260929 10:34:47.727900 26958 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.126:43935 every 8 connection(s)
I20260929 10:34:47.728770 26891 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
I20260929 10:34:47.730798 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 26891
I20260929 10:34:47.730923 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/master-0/wal/instance
I20260929 10:34:47.731184 15689 external_mini_cluster.cc:1428] Setting key 80d61bda5b92dbb2fc251dd9873e8f7a
I20260929 10:34:47.731645 26959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:34:47.738094 26959 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce47f2b3f23d45c59e39437a5b3d02d5: Bootstrap starting.
Sep 29 10:34:47 dist-test-slave-vg89 krb5kdc[26447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678087, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:34:47.740888 26959 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce47f2b3f23d45c59e39437a5b3d02d5: Neither blocks nor log segments found. Creating new log.
I20260929 10:34:47.742254 26959 log.cc:824] T 00000000000000000000000000000000 P ce47f2b3f23d45c59e39437a5b3d02d5: Log is configured to *not* fsync() on all Append() calls
I20260929 10:34:47.744580 26959 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce47f2b3f23d45c59e39437a5b3d02d5: No bootstrap required, opened a new log
I20260929 10:34:47.745541 26962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:34:47.732599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47918 (local address 127.15.82.126:43935)
0929 10:34:47.733016 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0929 10:34:47.733036 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0929 10:34:47.733077 (+ 41us) server_negotiation.cc:407] Connection header received
0929 10:34:47.733850 (+ 773us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:34:47.733873 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:34:47.734283 (+ 410us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:34:47.734682 (+ 399us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:34:47.735371 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:34:47.736326 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:34:47.737190 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:34:47.737658 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:34:47.737828 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:34:47.740597 (+ 2769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:34:47.740603 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:34:47.740618 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:34:47.740648 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:34:47.742609 (+ 1961us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:34:47.743125 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:34:47.743130 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:34:47.743135 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:34:47.743254 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:34:47.743522 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:34:47.743525 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:34:47.743526 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:34:47.744001 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:34:47.744177 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0929 10:34:47.744401 (+ 224us) server_negotiation.cc:299] Negotiation successful
0929 10:34:47.744663 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":117,"threads_started":1}
I20260929 10:34:47.747697 26959 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce47f2b3f23d45c59e39437a5b3d02d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce47f2b3f23d45c59e39437a5b3d02d5" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 43935 } }
I20260929 10:34:47.747987 26959 raft_consensus.cc:374] T 00000000000000000000000000000000 P ce47f2b3f23d45c59e39437a5b3d02d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:34:47.748054 26959 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce47f2b3f23d45c59e39437a5b3d02d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce47f2b3f23d45c59e39437a5b3d02d5, State: Initialized, Role: FOLLOWER
I20260929 10:34:47.748413 26959 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce47f2b3f23d45c59e39437a5b3d02d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce47f2b3f23d45c59e39437a5b3d02d5" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 43935 } }
I20260929 10:34:47.748523 26959 raft_consensus.cc:388] T 00000000000000000000000000000000 P ce47f2b3f23d45c59e39437a5b3d02d5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:34:47.748565 26959 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce47f2b3f23d45c59e39437a5b3d02d5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:34:47.748632 26959 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce47f2b3f23d45c59e39437a5b3d02d5 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:34:47.749439 26959 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce47f2b3f23d45c59e39437a5b3d02d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce47f2b3f23d45c59e39437a5b3d02d5" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 43935 } }
I20260929 10:34:47.749714 26959 leader_election.cc:302] T 00000000000000000000000000000000 P ce47f2b3f23d45c59e39437a5b3d02d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce47f2b3f23d45c59e39437a5b3d02d5; no voters:
I20260929 10:34:47.749954 26959 leader_election.cc:288] T 00000000000000000000000000000000 P ce47f2b3f23d45c59e39437a5b3d02d5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:34:47.750061 26964 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce47f2b3f23d45c59e39437a5b3d02d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:34:47.750322 26964 raft_consensus.cc:686] T 00000000000000000000000000000000 P ce47f2b3f23d45c59e39437a5b3d02d5 [term 1 LEADER]: Becoming Leader. State: Replica: ce47f2b3f23d45c59e39437a5b3d02d5, State: Running, Role: LEADER
I20260929 10:34:47.750782 26964 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce47f2b3f23d45c59e39437a5b3d02d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce47f2b3f23d45c59e39437a5b3d02d5" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 43935 } }
I20260929 10:34:47.751202 26959 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce47f2b3f23d45c59e39437a5b3d02d5 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:34:47.752460 26966 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce47f2b3f23d45c59e39437a5b3d02d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce47f2b3f23d45c59e39437a5b3d02d5. Latest consensus state: current_term: 1 leader_uuid: "ce47f2b3f23d45c59e39437a5b3d02d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce47f2b3f23d45c59e39437a5b3d02d5" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 43935 } } }
I20260929 10:34:47.752583 26966 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce47f2b3f23d45c59e39437a5b3d02d5 [sys.catalog]: This master's current role is: LEADER
I20260929 10:34:47.752882 26965 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce47f2b3f23d45c59e39437a5b3d02d5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ce47f2b3f23d45c59e39437a5b3d02d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce47f2b3f23d45c59e39437a5b3d02d5" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 43935 } } }
I20260929 10:34:47.752995 26973 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:34:47.753022 26965 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce47f2b3f23d45c59e39437a5b3d02d5 [sys.catalog]: This master's current role is: LEADER
I20260929 10:34:47.753532 26959 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260929 10:34:47.758634 26973 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:34:47.765672 26973 catalog_manager.cc:1347] Generated new cluster ID: 619879dbae1f42769c1a3993b40862b7
I20260929 10:34:47.765733 26973 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:34:47.780763 26973 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:34:47.782013 26973 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:34:47.793625 26973 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ce47f2b3f23d45c59e39437a5b3d02d5: Generated new TSK 0
I20260929 10:34:47.794476 26973 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.82.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab.
Entry for principal kudu/127.15.82.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab.
I20260929 10:34:48.458113 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.82.84:51769/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.65:0
--local_ip_for_outbound_sockets=127.15.82.65
--webserver_interface=127.15.82.65
--webserver_port=0
--tserver_master_addrs=127.15.82.126:43935
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab
--principal=kudu/127.15.82.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:41699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:34:48.606904 27004 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:34:48.607270 27004 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:34:48.607343 27004 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:34:48.612448 27004 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:34:48.612509 27004 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:34:48.612587 27004 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.65
I20260929 10:34:48.619683 27004 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:41699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.15.82.84:51769/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab
--principal=kudu/127.15.82.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.82.65
--webserver_port=0
--tserver_master_addrs=127.15.82.126:43935
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27004
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.65
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:34:48.621090 27004 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:34:48.622376 27004 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:34:48.633378 27014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:34:48.633381 27011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:34:48.633378 27012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:34:48.634725 27004 server_base.cc:1034] running on GCE node
I20260929 10:34:48.635183 27004 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:34:48.635958 27004 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:34:48.637153 27004 hybrid_clock.cc:648] HybridClock initialized: now 1790678088637109 us; error 58 us; skew 500 ppm
Sep 29 10:34:48 dist-test-slave-vg89 krb5kdc[26447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678088, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:34:48.641036 27004 init.cc:377] Logged in from keytab as kudu/127.15.82.65@KRBTEST.COM (short username kudu)
I20260929 10:34:48.642241 27004 webserver.cc:460] Webserver started at http://127.15.82.65:43411/ using document root <none> and password file <none>
I20260929 10:34:48.643034 27004 fs_manager.cc:359] Metadata directory not provided
I20260929 10:34:48.643136 27004 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:34:48.643474 27004 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 10:34:48 dist-test-slave-vg89 krb5kdc[26447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678088, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.65@KRBTEST.COM for HTTP/127.15.82.84@KRBTEST.COM
I20260929 10:34:48.660499 27004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/ts-0/data/instance:
uuid: "54ab227d92ee4e7b9db881c4ab15ed1b"
format_stamp: "Formatted at 2026-09-29 10:34:48 on dist-test-slave-vg89"
server_key: "8ac1f2ff2fc558b08be6296fbe73ea32"
server_key_iv: "da3fa80478814a264a180e9a6e1021e7"
server_key_version: "kuduclusterkey@0"
I20260929 10:34:48.661118 27004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/ts-0/wal/instance:
uuid: "54ab227d92ee4e7b9db881c4ab15ed1b"
format_stamp: "Formatted at 2026-09-29 10:34:48 on dist-test-slave-vg89"
server_key: "8ac1f2ff2fc558b08be6296fbe73ea32"
server_key_iv: "da3fa80478814a264a180e9a6e1021e7"
server_key_version: "kuduclusterkey@0"
I20260929 10:34:48.665524 27004 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260929 10:34:48.690855 27021 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:34:48.704353 27004 fs_manager.cc:725] Time spent opening block manager: real 0.024s user 0.002s sys 0.001s
I20260929 10:34:48.704560 27004 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/ts-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/ts-0/wal
uuid: "54ab227d92ee4e7b9db881c4ab15ed1b"
format_stamp: "Formatted at 2026-09-29 10:34:48 on dist-test-slave-vg89"
server_key: "8ac1f2ff2fc558b08be6296fbe73ea32"
server_key_iv: "da3fa80478814a264a180e9a6e1021e7"
server_key_version: "kuduclusterkey@0"
I20260929 10:34:48.704746 27004 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:34:48.722296 27004 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:34:48.726469 27004 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:34:48.726739 27004 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:34:48.727523 27004 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:34:48.728652 27004 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:34:48.728761 27004 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:34:48.728889 27004 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:34:48.728972 27004 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:34:48.749246 27004 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.65:33091
I20260929 10:34:48.749434 27136 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.65:33091 every 8 connection(s)
I20260929 10:34:48.750716 27004 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
I20260929 10:34:48.759779 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 27004
I20260929 10:34:48.759903 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/ts-0/wal/instance
I20260929 10:34:48.760200 15689 external_mini_cluster.cc:1428] Setting key a0ebd8d505ef729aa1cc03459459c018
Sep 29 10:34:48 dist-test-slave-vg89 krb5kdc[26447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678088, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.65@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:34:48.770103 27139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:34:48.752990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.65:47689 (local address 127.15.82.126:43935)
0929 10:34:48.753281 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0929 10:34:48.753289 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 10:34:48.754160 (+ 871us) server_negotiation.cc:407] Connection header received
0929 10:34:48.755256 (+ 1096us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:34:48.755261 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:34:48.755327 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:34:48.755428 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:34:48.756577 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:34:48.757160 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:34:48.758307 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:34:48.758625 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:34:48.758739 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:34:48.764942 (+ 6203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:34:48.764945 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:34:48.764949 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:34:48.764998 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:34:48.767210 (+ 2212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:34:48.767907 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:34:48.767913 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:34:48.767918 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:34:48.768021 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:34:48.768557 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:34:48.768562 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:34:48.768566 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:34:48.768849 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:34:48.768935 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0929 10:34:48.769726 (+ 791us) server_negotiation.cc:299] Negotiation successful
0929 10:34:48.769870 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":108,"threads_started":1}
WARNING: no policy specified for kudu/127.15.82.66@KRBTEST.COM; defaulting to no policy
I20260929 10:34:48.772126 27140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:34:48.753532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:43935 (local address 127.15.82.65:47689)
0929 10:34:48.754001 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0929 10:34:48.754056 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0929 10:34:48.754989 (+ 933us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:34:48.755610 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:34:48.755617 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:34:48.755945 (+ 328us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:34:48.756428 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:34:48.756442 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:34:48.757295 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:34:48.757298 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:34:48.758178 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:34:48.758187 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:34:48.758781 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:34:48.758786 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:34:48.758971 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:34:48.760696 (+ 1725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:34:48.760730 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:34:48.764758 (+ 4028us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:34:48.767368 (+ 2610us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:34:48.767375 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:34:48.767396 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:34:48.767778 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:34:48.768158 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:34:48.768163 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:34:48.768168 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:34:48.768431 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:34:48.769027 (+ 596us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:34:48.769035 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:34:48.769419 (+ 384us) client_negotiation.cc:769] Sending connection context
0929 10:34:48.770575 (+ 1156us) client_negotiation.cc:240] Negotiation successful
0929 10:34:48.770953 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":151,"threads_started":1}
I20260929 10:34:48.773458 27137 heartbeater.cc:347] Connected to a master server at 127.15.82.126:43935
I20260929 10:34:48.773698 27137 heartbeater.cc:464] Registering TS with master...
I20260929 10:34:48.774242 27137 heartbeater.cc:511] Master 127.15.82.126:43935 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.66@KRBTEST.COM" created.
I20260929 10:34:48.777493 26923 ts_manager.cc:194] Registered new tserver with Master: 54ab227d92ee4e7b9db881c4ab15ed1b (127.15.82.65:33091)
I20260929 10:34:48.778918 26923 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.82.65@KRBTEST.COM'} at 127.15.82.65:47689
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab.
Entry for principal kudu/127.15.82.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab.
I20260929 10:34:48.793262 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.82.84:51769/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.66:0
--local_ip_for_outbound_sockets=127.15.82.66
--webserver_interface=127.15.82.66
--webserver_port=0
--tserver_master_addrs=127.15.82.126:43935
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab
--principal=kudu/127.15.82.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:41699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 10:34:48 dist-test-slave-vg89 krb5kdc[26447](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790678088, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260929 10:34:48.944757 27143 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:34:48.945278 27143 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:34:48.945418 27143 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:34:48.951769 27143 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:34:48.951923 27143 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:34:48.952080 27143 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.66
I20260929 10:34:48.960450 27143 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:41699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.15.82.84:51769/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab
--principal=kudu/127.15.82.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.82.66
--webserver_port=0
--tserver_master_addrs=127.15.82.126:43935
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27143
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.66
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:34:48.962126 27143 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:34:48.963598 27143 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:34:48.976471 27150 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:34:48.976491 27152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:34:48.976969 27149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:34:48.977960 27143 server_base.cc:1034] running on GCE node
I20260929 10:34:48.978631 27143 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:34:48.979528 27143 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:34:48.980733 27143 hybrid_clock.cc:648] HybridClock initialized: now 1790678088980693 us; error 54 us; skew 500 ppm
Sep 29 10:34:48 dist-test-slave-vg89 krb5kdc[26447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678088, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:34:48.984719 27143 init.cc:377] Logged in from keytab as kudu/127.15.82.66@KRBTEST.COM (short username kudu)
I20260929 10:34:48.985915 27143 webserver.cc:460] Webserver started at http://127.15.82.66:39535/ using document root <none> and password file <none>
I20260929 10:34:48.986742 27143 fs_manager.cc:359] Metadata directory not provided
I20260929 10:34:48.986888 27143 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:34:48.987169 27143 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 10:34:48 dist-test-slave-vg89 krb5kdc[26447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678088, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.66@KRBTEST.COM for HTTP/127.15.82.84@KRBTEST.COM
I20260929 10:34:49.004833 27143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/ts-1/data/instance:
uuid: "4a4d0d759ce44f1ca15eaa48290e250c"
format_stamp: "Formatted at 2026-09-29 10:34:48 on dist-test-slave-vg89"
server_key: "5b6607a8ce75d2ec171bd5be128468b7"
server_key_iv: "e97776a8fc3d57ee2f8074dedb55425d"
server_key_version: "kuduclusterkey@0"
I20260929 10:34:49.005640 27143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/ts-1/wal/instance:
uuid: "4a4d0d759ce44f1ca15eaa48290e250c"
format_stamp: "Formatted at 2026-09-29 10:34:48 on dist-test-slave-vg89"
server_key: "5b6607a8ce75d2ec171bd5be128468b7"
server_key_iv: "e97776a8fc3d57ee2f8074dedb55425d"
server_key_version: "kuduclusterkey@0"
I20260929 10:34:49.016938 27143 fs_manager.cc:691] Time spent creating directory manager: real 0.011s user 0.007s sys 0.000s
I20260929 10:34:49.041774 27159 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:34:49.043601 27143 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260929 10:34:49.043751 27143 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/ts-1/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/ts-1/wal
uuid: "4a4d0d759ce44f1ca15eaa48290e250c"
format_stamp: "Formatted at 2026-09-29 10:34:48 on dist-test-slave-vg89"
server_key: "5b6607a8ce75d2ec171bd5be128468b7"
server_key_iv: "e97776a8fc3d57ee2f8074dedb55425d"
server_key_version: "kuduclusterkey@0"
I20260929 10:34:49.043900 27143 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:34:49.086961 27143 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:34:49.090677 27143 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:34:49.090890 27143 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:34:49.091583 27143 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:34:49.092510 27143 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:34:49.092582 27143 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:34:49.092648 27143 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:34:49.092702 27143 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:34:49.106055 27143 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.66:37373
I20260929 10:34:49.106070 27272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.66:37373 every 8 connection(s)
I20260929 10:34:49.107201 27143 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
I20260929 10:34:49.116413 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 27143
I20260929 10:34:49.116549 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/ts-1/wal/instance
I20260929 10:34:49.116861 15689 external_mini_cluster.cc:1428] Setting key 714c2d82e45ff8c63d31ff9438ae429d
Sep 29 10:34:49 dist-test-slave-vg89 krb5kdc[26447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678088, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.66@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:34:49.126952 27275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:34:49.109705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:43935 (local address 127.15.82.66:51735)
0929 10:34:49.111120 (+ 1415us) negotiation.cc:107] Waiting for socket to connect
0929 10:34:49.111173 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0929 10:34:49.112094 (+ 921us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:34:49.112675 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:34:49.112682 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:34:49.112951 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:34:49.113420 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:34:49.113452 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:34:49.114640 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:34:49.114644 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:34:49.116003 (+ 1359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:34:49.116014 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:34:49.116892 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:34:49.116896 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:34:49.117186 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:34:49.119564 (+ 2378us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:34:49.119586 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:34:49.121333 (+ 1747us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:34:49.123593 (+ 2260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:34:49.123600 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:34:49.123617 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:34:49.123969 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:34:49.124284 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:34:49.124287 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:34:49.124289 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:34:49.124477 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:34:49.124961 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:34:49.124968 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:34:49.125302 (+ 334us) client_negotiation.cc:769] Sending connection context
0929 10:34:49.125525 (+ 223us) client_negotiation.cc:240] Negotiation successful
0929 10:34:49.125872 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1177,"thread_start_us":146,"threads_started":1}
I20260929 10:34:49.127641 27139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:34:49.110921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:51735 (local address 127.15.82.126:43935)
0929 10:34:49.111041 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0929 10:34:49.111045 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:34:49.111274 (+ 229us) server_negotiation.cc:407] Connection header received
0929 10:34:49.112354 (+ 1080us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:34:49.112357 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:34:49.112412 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:34:49.112489 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:34:49.113583 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:34:49.114490 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:34:49.116147 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:34:49.116505 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:34:49.116588 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:34:49.121527 (+ 4939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:34:49.121531 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:34:49.121534 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:34:49.121563 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:34:49.123453 (+ 1890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:34:49.124090 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:34:49.124094 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:34:49.124096 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:34:49.124184 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:34:49.124586 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:34:49.124589 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:34:49.124591 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:34:49.124813 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:34:49.124908 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0929 10:34:49.127357 (+ 2449us) server_negotiation.cc:299] Negotiation successful
0929 10:34:49.127489 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260929 10:34:49.129262 27273 heartbeater.cc:347] Connected to a master server at 127.15.82.126:43935
I20260929 10:34:49.129572 27273 heartbeater.cc:464] Registering TS with master...
I20260929 10:34:49.130173 27273 heartbeater.cc:511] Master 127.15.82.126:43935 requested a full tablet report, sending...
I20260929 10:34:49.131356 26923 ts_manager.cc:194] Registered new tserver with Master: 4a4d0d759ce44f1ca15eaa48290e250c (127.15.82.66:37373)
WARNING: no policy specified for kudu/127.15.82.67@KRBTEST.COM; defaulting to no policy
I20260929 10:34:49.132103 26923 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.82.66@KRBTEST.COM'} at 127.15.82.66:51735
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab.
Entry for principal kudu/127.15.82.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab.
I20260929 10:34:49.155953 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.82.84:51769/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.67:0
--local_ip_for_outbound_sockets=127.15.82.67
--webserver_interface=127.15.82.67
--webserver_port=0
--tserver_master_addrs=127.15.82.126:43935
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab
--principal=kudu/127.15.82.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:41699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 10:34:49 dist-test-slave-vg89 krb5kdc[26447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678088, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.126@KRBTEST.COM for HTTP/127.15.82.84@KRBTEST.COM
W20260929 10:34:49.320209 27278 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:34:49.320621 27278 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:34:49.320695 27278 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:34:49.327080 27278 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:34:49.327171 27278 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:34:49.327275 27278 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.67
I20260929 10:34:49.335435 27278 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:41699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.15.82.84:51769/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab
--principal=kudu/127.15.82.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.82.67
--webserver_port=0
--tserver_master_addrs=127.15.82.126:43935
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27278
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.67
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:34:49.336876 27278 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:34:49.338297 27278 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:34:49.353398 27286 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:34:49.353544 27283 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:34:49.353936 27284 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:34:49.354864 27278 server_base.cc:1034] running on GCE node
I20260929 10:34:49.355327 27278 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:34:49.356102 27278 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:34:49.357327 27278 hybrid_clock.cc:648] HybridClock initialized: now 1790678089357286 us; error 52 us; skew 500 ppm
Sep 29 10:34:49 dist-test-slave-vg89 krb5kdc[26447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678089, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:34:49.360285 27278 init.cc:377] Logged in from keytab as kudu/127.15.82.67@KRBTEST.COM (short username kudu)
I20260929 10:34:49.361268 27278 webserver.cc:460] Webserver started at http://127.15.82.67:43601/ using document root <none> and password file <none>
I20260929 10:34:49.361924 27278 fs_manager.cc:359] Metadata directory not provided
I20260929 10:34:49.362016 27278 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:34:49.362247 27278 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 10:34:49 dist-test-slave-vg89 krb5kdc[26447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678089, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.67@KRBTEST.COM for HTTP/127.15.82.84@KRBTEST.COM
I20260929 10:34:49.378091 27278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/ts-2/data/instance:
uuid: "a4ea099caa014e85ab09d8321789e5d3"
format_stamp: "Formatted at 2026-09-29 10:34:49 on dist-test-slave-vg89"
server_key: "226b11f45cba960a17c41a5db4b09059"
server_key_iv: "41532342fa371c4db09062467d7d7b1c"
server_key_version: "kuduclusterkey@0"
I20260929 10:34:49.379390 27278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/ts-2/wal/instance:
uuid: "a4ea099caa014e85ab09d8321789e5d3"
format_stamp: "Formatted at 2026-09-29 10:34:49 on dist-test-slave-vg89"
server_key: "226b11f45cba960a17c41a5db4b09059"
server_key_iv: "41532342fa371c4db09062467d7d7b1c"
server_key_version: "kuduclusterkey@0"
I20260929 10:34:49.383627 27278 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260929 10:34:49.406859 27293 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:34:49.408509 27278 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.002s sys 0.001s
I20260929 10:34:49.408680 27278 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/ts-2/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/ts-2/wal
uuid: "a4ea099caa014e85ab09d8321789e5d3"
format_stamp: "Formatted at 2026-09-29 10:34:49 on dist-test-slave-vg89"
server_key: "226b11f45cba960a17c41a5db4b09059"
server_key_iv: "41532342fa371c4db09062467d7d7b1c"
server_key_version: "kuduclusterkey@0"
I20260929 10:34:49.408857 27278 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:34:49.431592 27278 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:34:49.434828 27278 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:34:49.435062 27278 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:34:49.435727 27278 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:34:49.436621 27278 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:34:49.436691 27278 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:34:49.436755 27278 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:34:49.436786 27278 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:34:49.447629 27278 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.67:44359
I20260929 10:34:49.447649 27406 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.67:44359 every 8 connection(s)
I20260929 10:34:49.448719 27278 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
I20260929 10:34:49.455019 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 27278
I20260929 10:34:49.455160 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/ts-2/wal/instance
I20260929 10:34:49.455452 15689 external_mini_cluster.cc:1428] Setting key 08413bde7690bc203dee30779e9aba73
Sep 29 10:34:49 dist-test-slave-vg89 krb5kdc[26447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678089, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.67@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:34:49.463886 27139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:34:49.451234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.67:35123 (local address 127.15.82.126:43935)
0929 10:34:49.451416 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0929 10:34:49.451421 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:34:49.452467 (+ 1046us) server_negotiation.cc:407] Connection header received
0929 10:34:49.453638 (+ 1171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:34:49.453641 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:34:49.453700 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:34:49.453779 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:34:49.455060 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:34:49.455834 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:34:49.457498 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:34:49.457776 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:34:49.457837 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:34:49.460305 (+ 2468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:34:49.460308 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:34:49.460310 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:34:49.460333 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:34:49.461824 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:34:49.462315 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:34:49.462320 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:34:49.462322 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:34:49.462458 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:34:49.462802 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:34:49.462807 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:34:49.462809 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:34:49.462993 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:34:49.463093 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0929 10:34:49.463604 (+ 511us) server_negotiation.cc:299] Negotiation successful
0929 10:34:49.463720 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260929 10:34:49.465019 27409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:34:49.451713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:43935 (local address 127.15.82.67:35123)
0929 10:34:49.452303 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0929 10:34:49.452366 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0929 10:34:49.453418 (+ 1052us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:34:49.453958 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:34:49.453965 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:34:49.454230 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:34:49.454873 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:34:49.454894 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:34:49.456005 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:34:49.456010 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:34:49.457369 (+ 1359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:34:49.457382 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:34:49.458028 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:34:49.458030 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:34:49.458172 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:34:49.458791 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:34:49.458808 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:34:49.460135 (+ 1327us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:34:49.461934 (+ 1799us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:34:49.461940 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:34:49.461956 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:34:49.462212 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:34:49.462552 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:34:49.462556 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:34:49.462558 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:34:49.462707 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:34:49.463103 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:34:49.463111 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:34:49.463365 (+ 254us) client_negotiation.cc:769] Sending connection context
0929 10:34:49.463554 (+ 189us) client_negotiation.cc:240] Negotiation successful
0929 10:34:49.463879 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":168,"threads_started":1}
I20260929 10:34:49.466166 27407 heartbeater.cc:347] Connected to a master server at 127.15.82.126:43935
I20260929 10:34:49.466457 27407 heartbeater.cc:464] Registering TS with master...
I20260929 10:34:49.466974 27407 heartbeater.cc:511] Master 127.15.82.126:43935 requested a full tablet report, sending...
I20260929 10:34:49.468272 26923 ts_manager.cc:194] Registered new tserver with Master: a4ea099caa014e85ab09d8321789e5d3 (127.15.82.67:44359)
I20260929 10:34:49.468880 26923 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.82.67@KRBTEST.COM'} at 127.15.82.67:35123
I20260929 10:34:49.469942 15689 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 10:34:49.480579 27139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:34:49.472258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47934 (local address 127.15.82.126:43935)
0929 10:34:49.472376 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0929 10:34:49.472381 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:34:49.473413 (+ 1032us) server_negotiation.cc:407] Connection header received
0929 10:34:49.473558 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:34:49.473562 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:34:49.473610 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:34:49.473693 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:34:49.474388 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:34:49.474924 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:34:49.475761 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:34:49.475983 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:34:49.476089 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:34:49.477459 (+ 1370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:34:49.477462 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:34:49.477465 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:34:49.477496 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:34:49.478958 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:34:49.479378 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:34:49.479384 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:34:49.479386 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:34:49.479463 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:34:49.479731 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:34:49.479736 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:34:49.479738 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:34:49.479906 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:34:49.480041 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0929 10:34:49.480271 (+ 230us) server_negotiation.cc:299] Negotiation successful
0929 10:34:49.480393 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260929 10:34:49.483647 26923 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47934:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 10:34:49.487967 26923 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:34:49.508736 27422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:34:49.497960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39022 (local address 127.15.82.67:44359)
0929 10:34:49.498318 (+ 358us) server_negotiation.cc:206] Beginning negotiation
0929 10:34:49.498330 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 10:34:49.502705 (+ 4375us) server_negotiation.cc:407] Connection header received
0929 10:34:49.502896 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:34:49.502905 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:34:49.503123 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:34:49.503277 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:34:49.503837 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:34:49.505072 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:34:49.506906 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:34:49.507736 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:34:49.507826 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:34:49.507999 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0929 10:34:49.508465 (+ 466us) server_negotiation.cc:299] Negotiation successful
0929 10:34:49.508572 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":175,"threads_started":1}
I20260929 10:34:49.509295 27420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:34:49.497191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.67:44359 (local address 127.0.0.1:39022)
0929 10:34:49.502573 (+ 5382us) negotiation.cc:107] Waiting for socket to connect
0929 10:34:49.502588 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 10:34:49.502707 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:34:49.503573 (+ 866us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:34:49.503576 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:34:49.503595 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:34:49.503691 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:34:49.503697 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:34:49.505216 (+ 1519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:34:49.505220 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:34:49.506725 (+ 1505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:34:49.506736 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:34:49.508109 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:34:49.508112 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:34:49.508232 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:34:49.508275 (+ 43us) client_negotiation.cc:769] Sending connection context
0929 10:34:49.509013 (+ 738us) client_negotiation.cc:240] Negotiation successful
0929 10:34:49.509153 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5308,"mutex_wait_us":1,"thread_start_us":72,"threads_started":1}
I20260929 10:34:49.512866 27341 tablet_service.cc:1467] Processing CreateTablet for tablet ab5f4ca32ea743acbd720159f5306187 (DEFAULT_TABLE table=test-table [id=db7787f887f94714a5607a81a8276be5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:34:49.514261 27341 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab5f4ca32ea743acbd720159f5306187. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:34:49.514569 27419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:34:49.498944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.65:33091 (local address 127.0.0.1:40742)
0929 10:34:49.502612 (+ 3668us) negotiation.cc:107] Waiting for socket to connect
0929 10:34:49.502624 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:34:49.502743 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:34:49.503457 (+ 714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:34:49.503464 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:34:49.503488 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:34:49.503667 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:34:49.503674 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:34:49.505831 (+ 2157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:34:49.505835 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:34:49.512149 (+ 6314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:34:49.512159 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:34:49.514149 (+ 1990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:34:49.514152 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:34:49.514248 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:34:49.514272 (+ 24us) client_negotiation.cc:769] Sending connection context
0929 10:34:49.514339 (+ 67us) client_negotiation.cc:240] Negotiation successful
0929 10:34:49.514432 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3610,"mutex_wait_us":5132,"spinlock_wait_cycles":4352,"thread_start_us":104,"threads_started":1}
I20260929 10:34:49.515089 27424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:34:49.499343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40742 (local address 127.15.82.65:33091)
0929 10:34:49.499697 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0929 10:34:49.499703 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 10:34:49.502767 (+ 3064us) server_negotiation.cc:407] Connection header received
0929 10:34:49.502886 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:34:49.502892 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:34:49.503123 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:34:49.503273 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:34:49.504600 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:34:49.505698 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:34:49.512999 (+ 7301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:34:49.513791 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:34:49.513914 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:34:49.514087 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0929 10:34:49.514853 (+ 766us) server_negotiation.cc:299] Negotiation successful
0929 10:34:49.514957 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":71,"threads_started":1}
I20260929 10:34:49.516088 27423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:34:49.497111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.66:37373 (local address 127.0.0.1:33072)
0929 10:34:49.499289 (+ 2178us) negotiation.cc:107] Waiting for socket to connect
0929 10:34:49.499348 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0929 10:34:49.499474 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:34:49.503509 (+ 4035us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:34:49.503512 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:34:49.503534 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:34:49.503803 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:34:49.503808 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:34:49.508926 (+ 5118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:34:49.508930 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:34:49.510882 (+ 1952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:34:49.510892 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:34:49.515731 (+ 4839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:34:49.515735 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:34:49.515825 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:34:49.515848 (+ 23us) client_negotiation.cc:769] Sending connection context
0929 10:34:49.515896 (+ 48us) client_negotiation.cc:240] Negotiation successful
0929 10:34:49.515960 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2000,"mutex_wait_us":170,"spinlock_wait_cycles":768,"thread_start_us":102,"threads_started":1}
I20260929 10:34:49.516963 27421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:34:49.497236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33072 (local address 127.15.82.66:37373)
0929 10:34:49.502989 (+ 5753us) server_negotiation.cc:206] Beginning negotiation
0929 10:34:49.503000 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 10:34:49.503021 (+ 21us) server_negotiation.cc:407] Connection header received
0929 10:34:49.503085 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:34:49.503089 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:34:49.503245 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:34:49.503378 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:34:49.506742 (+ 3364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:34:49.508708 (+ 1966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:34:49.511042 (+ 2334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:34:49.515310 (+ 4268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:34:49.516523 (+ 1213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:34:49.516683 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0929 10:34:49.516733 (+ 50us) server_negotiation.cc:299] Negotiation successful
0929 10:34:49.516826 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5653,"thread_start_us":125,"threads_started":1}
I20260929 10:34:49.520274 27071 tablet_service.cc:1467] Processing CreateTablet for tablet ab5f4ca32ea743acbd720159f5306187 (DEFAULT_TABLE table=test-table [id=db7787f887f94714a5607a81a8276be5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:34:49.521414 27071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab5f4ca32ea743acbd720159f5306187. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:34:49.521342 27207 tablet_service.cc:1467] Processing CreateTablet for tablet ab5f4ca32ea743acbd720159f5306187 (DEFAULT_TABLE table=test-table [id=db7787f887f94714a5607a81a8276be5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:34:49.522349 27207 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab5f4ca32ea743acbd720159f5306187. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:34:49.532645 27426 tablet_bootstrap.cc:492] T ab5f4ca32ea743acbd720159f5306187 P 54ab227d92ee4e7b9db881c4ab15ed1b: Bootstrap starting.
I20260929 10:34:49.533034 27425 tablet_bootstrap.cc:492] T ab5f4ca32ea743acbd720159f5306187 P 4a4d0d759ce44f1ca15eaa48290e250c: Bootstrap starting.
I20260929 10:34:49.534819 27427 tablet_bootstrap.cc:492] T ab5f4ca32ea743acbd720159f5306187 P a4ea099caa014e85ab09d8321789e5d3: Bootstrap starting.
I20260929 10:34:49.537534 27427 tablet_bootstrap.cc:654] T ab5f4ca32ea743acbd720159f5306187 P a4ea099caa014e85ab09d8321789e5d3: Neither blocks nor log segments found. Creating new log.
I20260929 10:34:49.538528 27427 log.cc:824] T ab5f4ca32ea743acbd720159f5306187 P a4ea099caa014e85ab09d8321789e5d3: Log is configured to *not* fsync() on all Append() calls
I20260929 10:34:49.540566 27426 tablet_bootstrap.cc:654] T ab5f4ca32ea743acbd720159f5306187 P 54ab227d92ee4e7b9db881c4ab15ed1b: Neither blocks nor log segments found. Creating new log.
I20260929 10:34:49.541527 27426 log.cc:824] T ab5f4ca32ea743acbd720159f5306187 P 54ab227d92ee4e7b9db881c4ab15ed1b: Log is configured to *not* fsync() on all Append() calls
I20260929 10:34:49.541925 27425 tablet_bootstrap.cc:654] T ab5f4ca32ea743acbd720159f5306187 P 4a4d0d759ce44f1ca15eaa48290e250c: Neither blocks nor log segments found. Creating new log.
I20260929 10:34:49.542903 27425 log.cc:824] T ab5f4ca32ea743acbd720159f5306187 P 4a4d0d759ce44f1ca15eaa48290e250c: Log is configured to *not* fsync() on all Append() calls
I20260929 10:34:49.545583 27426 tablet_bootstrap.cc:492] T ab5f4ca32ea743acbd720159f5306187 P 54ab227d92ee4e7b9db881c4ab15ed1b: No bootstrap required, opened a new log
I20260929 10:34:49.545850 27426 ts_tablet_manager.cc:1397] T ab5f4ca32ea743acbd720159f5306187 P 54ab227d92ee4e7b9db881c4ab15ed1b: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.000s
I20260929 10:34:49.550230 27426 raft_consensus.cc:348] T ab5f4ca32ea743acbd720159f5306187 P 54ab227d92ee4e7b9db881c4ab15ed1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a4d0d759ce44f1ca15eaa48290e250c" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 37373 } } peers { permanent_uuid: "a4ea099caa014e85ab09d8321789e5d3" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 44359 } } peers { permanent_uuid: "54ab227d92ee4e7b9db881c4ab15ed1b" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 33091 } }
I20260929 10:34:49.552896 27425 tablet_bootstrap.cc:492] T ab5f4ca32ea743acbd720159f5306187 P 4a4d0d759ce44f1ca15eaa48290e250c: No bootstrap required, opened a new log
I20260929 10:34:49.553182 27425 ts_tablet_manager.cc:1397] T ab5f4ca32ea743acbd720159f5306187 P 4a4d0d759ce44f1ca15eaa48290e250c: Time spent bootstrapping tablet: real 0.020s user 0.006s sys 0.000s
I20260929 10:34:49.554312 27426 raft_consensus.cc:374] T ab5f4ca32ea743acbd720159f5306187 P 54ab227d92ee4e7b9db881c4ab15ed1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:34:49.554443 27426 raft_consensus.cc:729] T ab5f4ca32ea743acbd720159f5306187 P 54ab227d92ee4e7b9db881c4ab15ed1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54ab227d92ee4e7b9db881c4ab15ed1b, State: Initialized, Role: FOLLOWER
I20260929 10:34:49.555086 27426 consensus_queue.cc:260] T ab5f4ca32ea743acbd720159f5306187 P 54ab227d92ee4e7b9db881c4ab15ed1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a4d0d759ce44f1ca15eaa48290e250c" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 37373 } } peers { permanent_uuid: "a4ea099caa014e85ab09d8321789e5d3" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 44359 } } peers { permanent_uuid: "54ab227d92ee4e7b9db881c4ab15ed1b" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 33091 } }
I20260929 10:34:49.555673 27427 tablet_bootstrap.cc:492] T ab5f4ca32ea743acbd720159f5306187 P a4ea099caa014e85ab09d8321789e5d3: No bootstrap required, opened a new log
I20260929 10:34:49.555894 27427 ts_tablet_manager.cc:1397] T ab5f4ca32ea743acbd720159f5306187 P a4ea099caa014e85ab09d8321789e5d3: Time spent bootstrapping tablet: real 0.021s user 0.004s sys 0.000s
I20260929 10:34:49.557654 27426 ts_tablet_manager.cc:1428] T ab5f4ca32ea743acbd720159f5306187 P 54ab227d92ee4e7b9db881c4ab15ed1b: Time spent starting tablet: real 0.012s user 0.006s sys 0.000s
I20260929 10:34:49.558086 27137 heartbeater.cc:503] Master 127.15.82.126:43935 was elected leader, sending a full tablet report...
I20260929 10:34:49.557906 27425 raft_consensus.cc:348] T ab5f4ca32ea743acbd720159f5306187 P 4a4d0d759ce44f1ca15eaa48290e250c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a4d0d759ce44f1ca15eaa48290e250c" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 37373 } } peers { permanent_uuid: "a4ea099caa014e85ab09d8321789e5d3" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 44359 } } peers { permanent_uuid: "54ab227d92ee4e7b9db881c4ab15ed1b" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 33091 } }
I20260929 10:34:49.558166 27425 raft_consensus.cc:374] T ab5f4ca32ea743acbd720159f5306187 P 4a4d0d759ce44f1ca15eaa48290e250c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:34:49.558220 27425 raft_consensus.cc:729] T ab5f4ca32ea743acbd720159f5306187 P 4a4d0d759ce44f1ca15eaa48290e250c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a4d0d759ce44f1ca15eaa48290e250c, State: Initialized, Role: FOLLOWER
I20260929 10:34:49.558737 27425 consensus_queue.cc:260] T ab5f4ca32ea743acbd720159f5306187 P 4a4d0d759ce44f1ca15eaa48290e250c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a4d0d759ce44f1ca15eaa48290e250c" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 37373 } } peers { permanent_uuid: "a4ea099caa014e85ab09d8321789e5d3" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 44359 } } peers { permanent_uuid: "54ab227d92ee4e7b9db881c4ab15ed1b" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 33091 } }
I20260929 10:34:49.559721 27425 ts_tablet_manager.cc:1428] T ab5f4ca32ea743acbd720159f5306187 P 4a4d0d759ce44f1ca15eaa48290e250c: Time spent starting tablet: real 0.006s user 0.002s sys 0.003s
I20260929 10:34:49.559764 27427 raft_consensus.cc:348] T ab5f4ca32ea743acbd720159f5306187 P a4ea099caa014e85ab09d8321789e5d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a4d0d759ce44f1ca15eaa48290e250c" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 37373 } } peers { permanent_uuid: "a4ea099caa014e85ab09d8321789e5d3" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 44359 } } peers { permanent_uuid: "54ab227d92ee4e7b9db881c4ab15ed1b" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 33091 } }
I20260929 10:34:49.560030 27427 raft_consensus.cc:374] T ab5f4ca32ea743acbd720159f5306187 P a4ea099caa014e85ab09d8321789e5d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:34:49.560081 27427 raft_consensus.cc:729] T ab5f4ca32ea743acbd720159f5306187 P a4ea099caa014e85ab09d8321789e5d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4ea099caa014e85ab09d8321789e5d3, State: Initialized, Role: FOLLOWER
I20260929 10:34:49.560091 27273 heartbeater.cc:503] Master 127.15.82.126:43935 was elected leader, sending a full tablet report...
I20260929 10:34:49.560515 27427 consensus_queue.cc:260] T ab5f4ca32ea743acbd720159f5306187 P a4ea099caa014e85ab09d8321789e5d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a4d0d759ce44f1ca15eaa48290e250c" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 37373 } } peers { permanent_uuid: "a4ea099caa014e85ab09d8321789e5d3" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 44359 } } peers { permanent_uuid: "54ab227d92ee4e7b9db881c4ab15ed1b" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 33091 } }
I20260929 10:34:49.561409 27427 ts_tablet_manager.cc:1428] T ab5f4ca32ea743acbd720159f5306187 P a4ea099caa014e85ab09d8321789e5d3: Time spent starting tablet: real 0.005s user 0.003s sys 0.003s
I20260929 10:34:49.561751 27407 heartbeater.cc:503] Master 127.15.82.126:43935 was elected leader, sending a full tablet report...
Sep 29 10:34:49 dist-test-slave-vg89 krb5kdc[26447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678088, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.126@KRBTEST.COM for HTTP/127.15.82.84@KRBTEST.COM
W20260929 10:34:49.608561 27274 tablet.cc:2369] T ab5f4ca32ea743acbd720159f5306187 P 4a4d0d759ce44f1ca15eaa48290e250c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 10:34:49.635066 27433 raft_consensus.cc:482] T ab5f4ca32ea743acbd720159f5306187 P 4a4d0d759ce44f1ca15eaa48290e250c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:34:49.635347 27433 raft_consensus.cc:504] T ab5f4ca32ea743acbd720159f5306187 P 4a4d0d759ce44f1ca15eaa48290e250c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a4d0d759ce44f1ca15eaa48290e250c" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 37373 } } peers { permanent_uuid: "a4ea099caa014e85ab09d8321789e5d3" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 44359 } } peers { permanent_uuid: "54ab227d92ee4e7b9db881c4ab15ed1b" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 33091 } }
I20260929 10:34:49.636919 27433 leader_election.cc:288] T ab5f4ca32ea743acbd720159f5306187 P 4a4d0d759ce44f1ca15eaa48290e250c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a4ea099caa014e85ab09d8321789e5d3 (127.15.82.67:44359), 54ab227d92ee4e7b9db881c4ab15ed1b (127.15.82.65:33091)
I20260929 10:34:49.652617 27422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:34:49.639229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:55143 (local address 127.15.82.67:44359)
0929 10:34:49.639395 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0929 10:34:49.639401 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 10:34:49.647229 (+ 7828us) server_negotiation.cc:407] Connection header received
0929 10:34:49.647301 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:34:49.647304 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:34:49.647376 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:34:49.647454 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:34:49.647984 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:34:49.648796 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:34:49.650661 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:34:49.651299 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:34:49.651404 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:34:49.651466 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0929 10:34:49.651861 (+ 395us) server_negotiation.cc:299] Negotiation successful
0929 10:34:49.651984 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 10:34:49.653613 27437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:34:49.638705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.67:44359 (local address 127.15.82.66:55143)
0929 10:34:49.647089 (+ 8384us) negotiation.cc:107] Waiting for socket to connect
0929 10:34:49.647106 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 10:34:49.647217 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:34:49.647579 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:34:49.647582 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:34:49.647602 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:34:49.647818 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:34:49.647828 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:34:49.649030 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:34:49.649034 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:34:49.650508 (+ 1474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:34:49.650519 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:34:49.651579 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:34:49.651583 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:34:49.651691 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:34:49.651717 (+ 26us) client_negotiation.cc:769] Sending connection context
0929 10:34:49.653296 (+ 1579us) client_negotiation.cc:240] Negotiation successful
0929 10:34:49.653434 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8307,"thread_start_us":167,"threads_started":1}
I20260929 10:34:49.654608 27351 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab5f4ca32ea743acbd720159f5306187" candidate_uuid: "4a4d0d759ce44f1ca15eaa48290e250c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4ea099caa014e85ab09d8321789e5d3" is_pre_election: true
I20260929 10:34:49.654978 27351 raft_consensus.cc:2454] T ab5f4ca32ea743acbd720159f5306187 P a4ea099caa014e85ab09d8321789e5d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a4d0d759ce44f1ca15eaa48290e250c in term 0.
I20260929 10:34:49.655681 27163 leader_election.cc:302] T ab5f4ca32ea743acbd720159f5306187 P 4a4d0d759ce44f1ca15eaa48290e250c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a4d0d759ce44f1ca15eaa48290e250c, a4ea099caa014e85ab09d8321789e5d3; no voters:
I20260929 10:34:49.656059 27433 raft_consensus.cc:2781] T ab5f4ca32ea743acbd720159f5306187 P 4a4d0d759ce44f1ca15eaa48290e250c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:34:49.656141 27433 raft_consensus.cc:482] T ab5f4ca32ea743acbd720159f5306187 P 4a4d0d759ce44f1ca15eaa48290e250c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:34:49.656184 27433 raft_consensus.cc:3037] T ab5f4ca32ea743acbd720159f5306187 P 4a4d0d759ce44f1ca15eaa48290e250c [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:34:49.657601 27433 raft_consensus.cc:504] T ab5f4ca32ea743acbd720159f5306187 P 4a4d0d759ce44f1ca15eaa48290e250c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a4d0d759ce44f1ca15eaa48290e250c" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 37373 } } peers { permanent_uuid: "a4ea099caa014e85ab09d8321789e5d3" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 44359 } } peers { permanent_uuid: "54ab227d92ee4e7b9db881c4ab15ed1b" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 33091 } }
I20260929 10:34:49.658057 27433 leader_election.cc:288] T ab5f4ca32ea743acbd720159f5306187 P 4a4d0d759ce44f1ca15eaa48290e250c [CANDIDATE]: Term 1 election: Requested vote from peers a4ea099caa014e85ab09d8321789e5d3 (127.15.82.67:44359), 54ab227d92ee4e7b9db881c4ab15ed1b (127.15.82.65:33091)
I20260929 10:34:49.659147 27351 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab5f4ca32ea743acbd720159f5306187" candidate_uuid: "4a4d0d759ce44f1ca15eaa48290e250c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4ea099caa014e85ab09d8321789e5d3"
I20260929 10:34:49.659292 27351 raft_consensus.cc:3037] T ab5f4ca32ea743acbd720159f5306187 P a4ea099caa014e85ab09d8321789e5d3 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:34:49.660692 27351 raft_consensus.cc:2454] T ab5f4ca32ea743acbd720159f5306187 P a4ea099caa014e85ab09d8321789e5d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a4d0d759ce44f1ca15eaa48290e250c in term 1.
I20260929 10:34:49.661140 27163 leader_election.cc:302] T ab5f4ca32ea743acbd720159f5306187 P 4a4d0d759ce44f1ca15eaa48290e250c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a4d0d759ce44f1ca15eaa48290e250c, a4ea099caa014e85ab09d8321789e5d3; no voters:
I20260929 10:34:49.661401 27433 raft_consensus.cc:2781] T ab5f4ca32ea743acbd720159f5306187 P 4a4d0d759ce44f1ca15eaa48290e250c [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:34:49.661646 27433 raft_consensus.cc:686] T ab5f4ca32ea743acbd720159f5306187 P 4a4d0d759ce44f1ca15eaa48290e250c [term 1 LEADER]: Becoming Leader. State: Replica: 4a4d0d759ce44f1ca15eaa48290e250c, State: Running, Role: LEADER
I20260929 10:34:49.662024 27433 consensus_queue.cc:237] T ab5f4ca32ea743acbd720159f5306187 P 4a4d0d759ce44f1ca15eaa48290e250c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a4d0d759ce44f1ca15eaa48290e250c" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 37373 } } peers { permanent_uuid: "a4ea099caa014e85ab09d8321789e5d3" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 44359 } } peers { permanent_uuid: "54ab227d92ee4e7b9db881c4ab15ed1b" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 33091 } }
I20260929 10:34:49.663196 27424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:34:49.637843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:45999 (local address 127.15.82.65:33091)
0929 10:34:49.638003 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0929 10:34:49.638009 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 10:34:49.642088 (+ 4079us) server_negotiation.cc:407] Connection header received
0929 10:34:49.642390 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:34:49.642425 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:34:49.642496 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:34:49.642553 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:34:49.644135 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:34:49.645011 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:34:49.646653 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:34:49.647319 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:34:49.647429 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:34:49.647493 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0929 10:34:49.662898 (+ 15405us) server_negotiation.cc:299] Negotiation successful
0929 10:34:49.663034 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260929 10:34:49.663658 27436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:34:49.637419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.65:33091 (local address 127.15.82.66:45999)
0929 10:34:49.638240 (+ 821us) negotiation.cc:107] Waiting for socket to connect
0929 10:34:49.638259 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 10:34:49.642214 (+ 3955us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:34:49.643619 (+ 1405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:34:49.643623 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:34:49.643822 (+ 199us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:34:49.643961 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:34:49.643971 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:34:49.645179 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:34:49.645183 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:34:49.646502 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:34:49.646514 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:34:49.662548 (+ 16034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:34:49.662552 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:34:49.662692 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:34:49.662734 (+ 42us) client_negotiation.cc:769] Sending connection context
0929 10:34:49.663424 (+ 690us) client_negotiation.cc:240] Negotiation successful
0929 10:34:49.663516 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":690,"thread_start_us":89,"threads_started":1}
I20260929 10:34:49.664393 27077 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab5f4ca32ea743acbd720159f5306187" candidate_uuid: "4a4d0d759ce44f1ca15eaa48290e250c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54ab227d92ee4e7b9db881c4ab15ed1b" is_pre_election: true
I20260929 10:34:49.664705 27077 raft_consensus.cc:2454] T ab5f4ca32ea743acbd720159f5306187 P 54ab227d92ee4e7b9db881c4ab15ed1b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a4d0d759ce44f1ca15eaa48290e250c in term 0.
I20260929 10:34:49.665055 27076 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab5f4ca32ea743acbd720159f5306187" candidate_uuid: "4a4d0d759ce44f1ca15eaa48290e250c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54ab227d92ee4e7b9db881c4ab15ed1b"
I20260929 10:34:49.665169 27076 raft_consensus.cc:3037] T ab5f4ca32ea743acbd720159f5306187 P 54ab227d92ee4e7b9db881c4ab15ed1b [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:34:49.666875 27076 raft_consensus.cc:2454] T ab5f4ca32ea743acbd720159f5306187 P 54ab227d92ee4e7b9db881c4ab15ed1b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a4d0d759ce44f1ca15eaa48290e250c in term 1.
I20260929 10:34:49.667088 26922 catalog_manager.cc:5495] T ab5f4ca32ea743acbd720159f5306187 P 4a4d0d759ce44f1ca15eaa48290e250c reported cstate change: term changed from 0 to 1, leader changed from <none> to 4a4d0d759ce44f1ca15eaa48290e250c (127.15.82.66). New cstate: current_term: 1 leader_uuid: "4a4d0d759ce44f1ca15eaa48290e250c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a4d0d759ce44f1ca15eaa48290e250c" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 37373 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4ea099caa014e85ab09d8321789e5d3" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 44359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54ab227d92ee4e7b9db881c4ab15ed1b" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 33091 } health_report { overall_health: UNKNOWN } } }
I20260929 10:34:49.695588 27421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:34:49.690008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33076 (local address 127.15.82.66:37373)
0929 10:34:49.690179 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0929 10:34:49.690184 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:34:49.690201 (+ 17us) server_negotiation.cc:407] Connection header received
0929 10:34:49.690295 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:34:49.690298 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:34:49.690374 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:34:49.690522 (+ 148us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:34:49.690958 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:34:49.691836 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:34:49.693212 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:34:49.693565 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:34:49.693673 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:34:49.694155 (+ 482us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:34:49.694796 (+ 641us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:34:49.694920 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0929 10:34:49.695367 (+ 447us) server_negotiation.cc:299] Negotiation successful
0929 10:34:49.695434 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260929 10:34:49.703300 27408 tablet.cc:2369] T ab5f4ca32ea743acbd720159f5306187 P a4ea099caa014e85ab09d8321789e5d3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 10:34:49.703346 27076 raft_consensus.cc:1261] T ab5f4ca32ea743acbd720159f5306187 P 54ab227d92ee4e7b9db881c4ab15ed1b [term 1 FOLLOWER]: Refusing update from remote peer 4a4d0d759ce44f1ca15eaa48290e250c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:34:49.704105 27438 consensus_queue.cc:1035] T ab5f4ca32ea743acbd720159f5306187 P 4a4d0d759ce44f1ca15eaa48290e250c [LEADER]: Connected to new peer: Peer: permanent_uuid: "54ab227d92ee4e7b9db881c4ab15ed1b" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 33091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:34:49.704401 27351 raft_consensus.cc:1261] T ab5f4ca32ea743acbd720159f5306187 P a4ea099caa014e85ab09d8321789e5d3 [term 1 FOLLOWER]: Refusing update from remote peer 4a4d0d759ce44f1ca15eaa48290e250c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:34:49.705156 27438 consensus_queue.cc:1035] T ab5f4ca32ea743acbd720159f5306187 P 4a4d0d759ce44f1ca15eaa48290e250c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4ea099caa014e85ab09d8321789e5d3" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 44359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:34:49.715018 27441 mvcc.cc:205] Tried to move back new op lower bound from 7334617455415111680 to 7334617455260049408. Current Snapshot: MvccSnapshot[applied={T|T < 7334617455415111680}]
I20260929 10:34:49.720733 27442 mvcc.cc:205] Tried to move back new op lower bound from 7334617455415111680 to 7334617455260049408. Current Snapshot: MvccSnapshot[applied={T|T < 7334617455415111680}]
I20260929 10:34:49.723424 27443 mvcc.cc:205] Tried to move back new op lower bound from 7334617455415111680 to 7334617455260049408. Current Snapshot: MvccSnapshot[applied={T|T < 7334617455415111680}]
I20260929 10:34:49.731773 26922 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47934:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:34:49.731941 26922 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47934:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:34:49.735069 26922 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ce47f2b3f23d45c59e39437a5b3d02d5: Sending DeleteTablet for 3 replicas of tablet ab5f4ca32ea743acbd720159f5306187
I20260929 10:34:49.736675 27207 tablet_service.cc:1514] Processing DeleteTablet for tablet ab5f4ca32ea743acbd720159f5306187 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:34:49 UTC) from {username='kudu', principal='kudu/127.15.82.126@KRBTEST.COM'} at 127.0.0.1:33072
I20260929 10:34:49.737190 27341 tablet_service.cc:1514] Processing DeleteTablet for tablet ab5f4ca32ea743acbd720159f5306187 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:34:49 UTC) from {username='kudu', principal='kudu/127.15.82.126@KRBTEST.COM'} at 127.0.0.1:39022
I20260929 10:34:49.737648 27071 tablet_service.cc:1514] Processing DeleteTablet for tablet ab5f4ca32ea743acbd720159f5306187 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:34:49 UTC) from {username='kudu', principal='kudu/127.15.82.126@KRBTEST.COM'} at 127.0.0.1:40742
I20260929 10:34:49.738767 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 27004
I20260929 10:34:49.758672 27451 tablet_replica.cc:330] T ab5f4ca32ea743acbd720159f5306187 P 4a4d0d759ce44f1ca15eaa48290e250c: stopping tablet replica
I20260929 10:34:49.759058 27452 tablet_replica.cc:330] T ab5f4ca32ea743acbd720159f5306187 P a4ea099caa014e85ab09d8321789e5d3: stopping tablet replica
I20260929 10:34:49.763738 27452 raft_consensus.cc:2229] T ab5f4ca32ea743acbd720159f5306187 P a4ea099caa014e85ab09d8321789e5d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:34:49.764271 27451 raft_consensus.cc:2229] T ab5f4ca32ea743acbd720159f5306187 P 4a4d0d759ce44f1ca15eaa48290e250c [term 1 LEADER]: Raft consensus shutting down.
I20260929 10:34:49.765717 27451 raft_consensus.cc:2258] T ab5f4ca32ea743acbd720159f5306187 P 4a4d0d759ce44f1ca15eaa48290e250c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:34:49.766185 27452 raft_consensus.cc:2258] T ab5f4ca32ea743acbd720159f5306187 P a4ea099caa014e85ab09d8321789e5d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:34:49.768046 27452 ts_tablet_manager.cc:1905] T ab5f4ca32ea743acbd720159f5306187 P a4ea099caa014e85ab09d8321789e5d3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:34:49.769418 27451 ts_tablet_manager.cc:1905] T ab5f4ca32ea743acbd720159f5306187 P 4a4d0d759ce44f1ca15eaa48290e250c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:34:49.772301 27451 ts_tablet_manager.cc:1918] T ab5f4ca32ea743acbd720159f5306187 P 4a4d0d759ce44f1ca15eaa48290e250c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:34:49.775025 27452 ts_tablet_manager.cc:1918] T ab5f4ca32ea743acbd720159f5306187 P a4ea099caa014e85ab09d8321789e5d3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:34:49.779785 27452 log.cc:1196] T ab5f4ca32ea743acbd720159f5306187 P a4ea099caa014e85ab09d8321789e5d3: Deleting WAL directory at /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/ts-2/wal/wals/ab5f4ca32ea743acbd720159f5306187
I20260929 10:34:49.779953 27451 log.cc:1196] T ab5f4ca32ea743acbd720159f5306187 P 4a4d0d759ce44f1ca15eaa48290e250c: Deleting WAL directory at /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790677981592896-15689-0/minicluster-data/ts-1/wal/wals/ab5f4ca32ea743acbd720159f5306187
I20260929 10:34:49.780391 27452 ts_tablet_manager.cc:1939] T ab5f4ca32ea743acbd720159f5306187 P a4ea099caa014e85ab09d8321789e5d3: Deleting consensus metadata
I20260929 10:34:49.780771 27451 ts_tablet_manager.cc:1939] T ab5f4ca32ea743acbd720159f5306187 P 4a4d0d759ce44f1ca15eaa48290e250c: Deleting consensus metadata
W20260929 10:34:49.781206 26907 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.82.65:33091 (error 108)
I20260929 10:34:49.783450 26910 catalog_manager.cc:4841] TS a4ea099caa014e85ab09d8321789e5d3 (127.15.82.67:44359): tablet ab5f4ca32ea743acbd720159f5306187 (table test-table [id=db7787f887f94714a5607a81a8276be5]) successfully deleted
I20260929 10:34:49.784798 26909 catalog_manager.cc:4841] TS 4a4d0d759ce44f1ca15eaa48290e250c (127.15.82.66:37373): tablet ab5f4ca32ea743acbd720159f5306187 (table test-table [id=db7787f887f94714a5607a81a8276be5]) successfully deleted
I20260929 10:34:49.785135 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 27143
I20260929 10:34:49.788942 27423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:34:49.788576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.65:33091 (local address 127.0.0.1:40756)
0929 10:34:49.788772 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0929 10:34:49.788849 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.65:33091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
W20260929 10:34:49.789141 26907 catalog_manager.cc:4584] TS 54ab227d92ee4e7b9db881c4ab15ed1b (127.15.82.65:33091): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ab5f4ca32ea743acbd720159f5306187: Network error: Client connection negotiation failed: client connection to 127.15.82.65:33091: connect: Connection refused (error 111)
I20260929 10:34:49.797406 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 27278
I20260929 10:34:49.805058 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 26891
I20260929 10:34:49.813512 15689 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260929 10:34:50.159096 15689 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260929 10:34:50.159269 15689 mini_ranger.cc:66] Stopping Ranger...
I20260929 10:34:50.505163 15689 mini_ranger.cc:68] Stopped Ranger
2026-09-29 10:34:50.505 UTC [26477] LOG: received smart shutdown request
2026-09-29 10:34:50.509 UTC [26477] LOG: background worker "logical replication launcher" (PID 26486) exited with exit code 1
2026-09-29 10:34:50.509 UTC [26481] LOG: shutting down
2026-09-29 10:34:50.694 UTC [26477] LOG: database system is shut down
2026-09-29T10:34:50Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (47097 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:34:50 dist-test-slave-vg89 krb5kdc[27463](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:34:50 dist-test-slave-vg89 krb5kdc[27463](info): set up 2 sockets
Sep 29 10:34:50 dist-test-slave-vg89 krb5kdc[27463](info): commencing operation
krb5kdc: starting...
W20260929 10:34:51.866787 15689 mini_kdc.cc:121] Time spent starting KDC: real 1.033s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Sep 29 10:34:51 dist-test-slave-vg89 krb5kdc[27463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678091, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:34:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:34:51Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.15.82.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.15.82.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.15.82.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/krb5kdc/rangeradmin_127.15.82.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.15.82.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/krb5kdc/rangeradmin_127.15.82.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.15.82.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.15.82.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.15.82.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/krb5kdc/rangerlookup_127.15.82.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.15.82.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/krb5kdc/rangerlookup_127.15.82.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.15.82.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.15.82.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.15.82.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/krb5kdc/HTTP_127.15.82.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.15.82.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/krb5kdc/HTTP_127.15.82.84@KRBTEST.COM.keytab.
I20260929 10:34:52.041214 15689 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskiKMUyb/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/postgres -l logfile start
2026-09-29 10:34:54.559 UTC [27493] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-29 10:34:54.559 UTC [27493] LOG: listening on IPv4 address "127.15.82.84", port 39547
2026-09-29 10:34:54.562 UTC [27493] LOG: listening on Unix socket "/tmp/.s.PGSQL.39547"
2026-09-29 10:34:54.584 UTC [27496] LOG: database system was shut down at 2026-09-29 10:34:53 UTC
2026-09-29 10:34:54.589 UTC [27493] LOG: database system is ready to accept connections
I20260929 10:34:55.538585 15689 mini_postgres.cc:96] Postgres bound to 39547
2026-09-29 10:34:55.545 UTC [27504] FATAL: database "slave" does not exist
127.15.82.84:39547 - accepting connections
I20260929 10:34:55.546157 15689 mini_ranger.cc:161] Starting Ranger...
I20260929 10:34:55.561398 15689 mini_ranger.cc:84] Created miniranger Postgres user
I20260929 10:34:56.059870 15689 mini_ranger.cc:87] Created ranger Postgres database
I20260929 10:34:56.059957 15689 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ranger-admin
2026-09-29 10:34:56,331 [I] DB FLAVOR :POSTGRES
2026-09-29 10:34:56,332 [I] --------- Verifying Ranger DB connection ---------
2026-09-29 10:34:56,332 [I] Checking connection..
2026-09-29 10:34:56,332 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiKMUyb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskiKMUyb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.82.84:39547/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-29 10:34:56,581 [I] Checking connection passed.
2026-09-29 10:34:56,581 [I] --------- Verifying version history table ---------
2026-09-29 10:34:56,581 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiKMUyb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskiKMUyb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.82.84:39547/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 10:34:56,859 [I] Table x_db_version_h does not exist in database ranger
2026-09-29 10:34:56,859 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiKMUyb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskiKMUyb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.82.84:39547/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 10:34:57,126 [I] Table x_db_version_h does not exist in database ranger
2026-09-29 10:34:57,126 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-29 10:34:57,127 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiKMUyb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskiKMUyb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.82.84:39547/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskiKMUyb/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-29 10:34:57.364 UTC [27568] WARNING: there is no transaction in progress
2026-09-29 10:34:57,371 [I] create_dbversion_catalog.sql file imported successfully
2026-09-29 10:34:57,371 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiKMUyb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskiKMUyb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.82.84:39547/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 10:34:57,633 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-29 10:34:57,633 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-29 10:34:57,633 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiKMUyb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskiKMUyb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.82.84:39547/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-29 10:34:57,879 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiKMUyb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskiKMUyb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.82.84:39547/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-29 10:34:58,138 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiKMUyb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskiKMUyb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.82.84:39547/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-vg89.c.gcp-upstream.internal','N') ;"
2026-09-29 10:34:58,379 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiKMUyb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskiKMUyb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.82.84:39547/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-29 10:34:58,632 [I] Table x_portal_user does not exist in database ranger
2026-09-29 10:34:58,633 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiKMUyb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskiKMUyb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.82.84:39547/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-29 10:34:58,881 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-29 10:34:58,881 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiKMUyb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskiKMUyb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.82.84:39547/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-29 10:34:59,106 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiKMUyb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskiKMUyb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.82.84:39547/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-29 10:34:59,340 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-29 10:34:59,340 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiKMUyb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskiKMUyb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.82.84:39547/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskiKMUyb/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-29 10:35:00.301 UTC [27676] WARNING: there is no transaction in progress
2026-09-29 10:35:00.319 UTC [27676] WARNING: there is no transaction in progress
2026-09-29 10:35:00.336 UTC [27676] WARNING: there is no transaction in progress
2026-09-29 10:35:00.353 UTC [27676] WARNING: there is no transaction in progress
2026-09-29 10:35:00.370 UTC [27676] WARNING: there is no transaction in progress
2026-09-29 10:35:00.389 UTC [27676] WARNING: there is no transaction in progress
2026-09-29 10:35:00.445 UTC [27676] WARNING: there is no transaction in progress
2026-09-29 10:35:00.452 UTC [27676] WARNING: there is no transaction in progress
2026-09-29 10:35:00.467 UTC [27676] WARNING: there is no transaction in progress
2026-09-29 10:35:00.477 UTC [27676] WARNING: there is no transaction in progress
2026-09-29 10:35:00.487 UTC [27676] WARNING: there is no transaction in progress
2026-09-29 10:35:00.502 UTC [27676] WARNING: there is no transaction in progress
2026-09-29 10:35:00.512 UTC [27676] WARNING: there is no transaction in progress
2026-09-29 10:35:00.519 UTC [27676] WARNING: there is no transaction in progress
2026-09-29 10:35:00.527 UTC [27676] WARNING: there is no transaction in progress
2026-09-29 10:35:00.535 UTC [27676] WARNING: there is no transaction in progress
2026-09-29 10:35:01,184 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-29 10:35:01,185 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiKMUyb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskiKMUyb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.82.84:39547/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-29 10:35:01,433 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-29 10:35:01,434 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiKMUyb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskiKMUyb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.82.84:39547/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-29 10:35:01,669 [I] Table x_portal_user already exists in database 'ranger'
2026-09-29 10:35:01,670 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiKMUyb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskiKMUyb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.82.84:39547/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-29 10:35:01,932 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-29 10:35:01,932 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiKMUyb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskiKMUyb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.82.84:39547/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-29 10:35:02,160 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiKMUyb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskiKMUyb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.82.84:39547/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-29 10:35:02,420 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiKMUyb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskiKMUyb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.82.84:39547/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-vg89.c.gcp-upstream.internal';"
2026-09-29 10:35:02,647 [I] CORE_DB_SCHEMA import status has been updated
2026-09-29 10:35:02,647 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiKMUyb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskiKMUyb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.82.84:39547/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-29 10:35:02,890 [I] DB_PATCHES have already been applied
I20260929 10:35:02.895560 15689 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ranger-admin:/tmp/dist-test-taskiKMUyb/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskiKMUyb/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskiKMUyb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskiKMUyb/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260929 10:35:02.895668 15689 mini_ranger.cc:193] Using host: 127.15.82.84
I20260929 10:35:02.899699 15689 mini_ranger.cc:227] Ranger admin URL: http://127.15.82.84:52773
Sep 29, 2026 10:35:03 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 29, 2026 10:35:03 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 29, 2026 10:35:03 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ranger-admin/ews/webapp
Sep 29, 2026 10:35:03 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 29, 2026 10:35:03 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 29, 2026 10:35:03 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ranger-admin/ews/webapp].
Sep 29, 2026 10:35:03 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260929 10:35:04.022346 15689 mini_ranger.cc:160] Time spent starting Ranger: real 8.476s user 0.000s sys 0.011s
Sep 29, 2026 10:35:24 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 29, 2026 10:35:24 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 29, 2026 10:35:24 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 29, 2026 10:35:24 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 29, 2026 10:35:24 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 29, 2026 10:35:24 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 29, 2026 10:35:24 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 29, 2026 10:35:24 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 29, 2026 10:35:24 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 29, 2026 10:35:24 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 29, 2026 10:35:24 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 29, 2026 10:35:24 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 29, 2026 10:35:24 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 29, 2026 10:35:24 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 29, 2026 10:35:24 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 29, 2026 10:35:24 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 29, 2026 10:35:24 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 29, 2026 10:35:25 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260929 10:35:25.898675 15689 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.15.82.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.15.82.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.15.82.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/krb5kdc/rangerkms_127.15.82.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.15.82.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/krb5kdc/rangerkms_127.15.82.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.15.82.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.15.82.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.15.82.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/krb5kdc/HTTP_127.15.82.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.15.82.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/krb5kdc/HTTP_127.15.82.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 29 10:35:25 dist-test-slave-vg89 krb5kdc[27463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678125, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260929 10:35:25.990787 15689 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260929 10:35:26.000962 15689 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260929 10:35:26.918373 15689 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260929 10:35:26.918489 15689 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ranger-kms
I20260929 10:35:26.918505 15689 mini_ranger_kms.cc:227] Using postgres at 127.15.82.84:39547
2026-09-29 10:35:27,113 [I] DB FLAVOR :POSTGRES
2026-09-29 10:35:27,113 [I] --------- Verifying Ranger DB connection ---------
2026-09-29 10:35:27,114 [I] Checking connection
2026-09-29 10:35:27,114 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiKMUyb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.82.84:39547/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-29 10:35:27,354 [I] connection success
2026-09-29 10:35:27,354 [I] --------- Verifying Ranger DB tables ---------
2026-09-29 10:35:27,354 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-29 10:35:27,354 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiKMUyb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.82.84:39547/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-29 10:35:27,602 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-29 10:35:27,602 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-29 10:35:27,602 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-29 10:35:27,602 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiKMUyb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.82.84:39547/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-29 10:35:27,854 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260929 10:35:28.327668 15689 mini_ranger_kms.cc:310] Created kms service
I20260929 10:35:28.542014 15689 mini_ranger_kms.cc:326] Created kudu user
I20260929 10:35:28.585485 15689 mini_ranger_kms.cc:343] Created rangerkms user
Sep 29 10:35:28 dist-test-slave-vg89 krb5kdc[27463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678125, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.15.82.84@KRBTEST.COM
I20260929 10:35:28.962253 15689 mini_ranger_kms.cc:384] Added ranger policy
I20260929 10:35:28.962539 15689 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ranger-kms:/tmp/dist-test-taskiKMUyb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskiKMUyb/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskiKMUyb/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260929 10:35:28.962579 15689 mini_ranger_kms.cc:240] Using host: 127.15.82.84
I20260929 10:35:28.967196 15689 mini_ranger_kms.cc:276] Ranger KMS PID: 27856
I20260929 10:35:28.967280 15689 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.15.82.84:34645
Sep 29, 2026 10:35:30 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 29, 2026 10:35:30 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 29, 2026 10:35:30 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 29, 2026 10:35:30 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 29, 2026 10:35:30 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ranger-kms/ews/webapp].
Sep 29, 2026 10:35:30 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260929 10:35:31.225301 15689 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.235s user 0.020s sys 0.158s
I20260929 10:35:31.225441 15689 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260929 10:35:31.225524 15689 mini_ranger_kms.cc:401] 127.15.82.84:34645/kms/v1/keys
Sep 29 10:35:31 dist-test-slave-vg89 krb5kdc[27463](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790678131, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.82.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 29 10:35:31 dist-test-slave-vg89 krb5kdc[27463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678131, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.82.84@KRBTEST.COM for HTTP/127.15.82.84@KRBTEST.COM
Sep 29 10:35:32 dist-test-slave-vg89 krb5kdc[27463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678131, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.82.84@KRBTEST.COM for HTTP/127.15.82.84@KRBTEST.COM
Sep 29 10:35:34 dist-test-slave-vg89 krb5kdc[27463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678134, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.15.82.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab.
Entry for principal kudu/127.15.82.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab.
I20260929 10:35:34.390815 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:33739
--webserver_interface=127.15.82.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab
--principal=kudu/127.15.82.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:44449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.82.126:33739
--ranger_config_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.82.84:34645/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:35:34.508390 27907 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:35:34.508663 27907 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:35:34.508702 27907 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:35:34.512837 27907 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:35:34.512902 27907 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:35:34.512928 27907 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:35:34.512945 27907 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:35:34.512995 27907 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:35:34.518869 27907 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:44449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/master-0/wal
--ranger_kms_url=127.15.82.84:34645/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.15.82.126:33739
--ranger_config_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab
--principal=kudu/127.15.82.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:33739
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.82.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27907
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:35:34.520004 27907 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:35:34.521051 27907 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:35:34.530704 27913 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:35:34.530768 27915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:35:34.530946 27907 server_base.cc:1034] running on GCE node
W20260929 10:35:34.530704 27912 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:35:34.531579 27907 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:35:34.532716 27907 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:35:34.533888 27907 hybrid_clock.cc:648] HybridClock initialized: now 1790678134533871 us; error 39 us; skew 500 ppm
Sep 29 10:35:34 dist-test-slave-vg89 krb5kdc[27463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678134, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:35:34.540912 27907 init.cc:377] Logged in from keytab as kudu/127.15.82.126@KRBTEST.COM (short username kudu)
I20260929 10:35:34.541815 27907 webserver.cc:460] Webserver started at http://127.15.82.126:36465/ using document root <none> and password file <none>
I20260929 10:35:34.542377 27907 fs_manager.cc:359] Metadata directory not provided
I20260929 10:35:34.542501 27907 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:35:34.542726 27907 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 10:35:34 dist-test-slave-vg89 krb5kdc[27463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678134, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.126@KRBTEST.COM for HTTP/127.15.82.84@KRBTEST.COM
I20260929 10:35:34.596076 27907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/master-0/data/instance:
uuid: "a751487001594e6293a13894cf876e50"
format_stamp: "Formatted at 2026-09-29 10:35:34 on dist-test-slave-vg89"
server_key: "3a5bc85ba5a60891e232a5e51a2ac65e"
server_key_iv: "c2504553238db51241ea80876b271619"
server_key_version: "kuduclusterkey@0"
I20260929 10:35:34.596694 27907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/master-0/wal/instance:
uuid: "a751487001594e6293a13894cf876e50"
format_stamp: "Formatted at 2026-09-29 10:35:34 on dist-test-slave-vg89"
server_key: "3a5bc85ba5a60891e232a5e51a2ac65e"
server_key_iv: "c2504553238db51241ea80876b271619"
server_key_version: "kuduclusterkey@0"
I20260929 10:35:34.600760 27907 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 10:35:34.618242 27922 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:35:34.619935 27907 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 10:35:34.620129 27907 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/master-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/master-0/wal
uuid: "a751487001594e6293a13894cf876e50"
format_stamp: "Formatted at 2026-09-29 10:35:34 on dist-test-slave-vg89"
server_key: "3a5bc85ba5a60891e232a5e51a2ac65e"
server_key_iv: "c2504553238db51241ea80876b271619"
server_key_version: "kuduclusterkey@0"
I20260929 10:35:34.620273 27907 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:35:34.645666 27907 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:35:34.648882 27907 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:35:34.649108 27907 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:35:34.657433 27907 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.126:33739
I20260929 10:35:34.657446 27974 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.126:33739 every 8 connection(s)
I20260929 10:35:34.658535 27907 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
I20260929 10:35:34.658993 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 27907
I20260929 10:35:34.659124 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/master-0/wal/instance
I20260929 10:35:34.659420 15689 external_mini_cluster.cc:1428] Setting key 1071e2718f8c22bbc8188fcf3000ec74
I20260929 10:35:34.661754 27975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:35:34.668551 27975 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a751487001594e6293a13894cf876e50: Bootstrap starting.
Sep 29 10:35:34 dist-test-slave-vg89 krb5kdc[27463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678134, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:35:34.671264 27975 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a751487001594e6293a13894cf876e50: Neither blocks nor log segments found. Creating new log.
I20260929 10:35:34.672617 27975 log.cc:824] T 00000000000000000000000000000000 P a751487001594e6293a13894cf876e50: Log is configured to *not* fsync() on all Append() calls
I20260929 10:35:34.674465 27978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:34.660957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52114 (local address 127.15.82.126:33739)
0929 10:35:34.661370 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0929 10:35:34.661388 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0929 10:35:34.661434 (+ 46us) server_negotiation.cc:407] Connection header received
0929 10:35:34.662293 (+ 859us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:35:34.662316 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:35:34.662765 (+ 449us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:35:34.663131 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:35:34.663806 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:34.665346 (+ 1540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:34.666183 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:34.666926 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:34.667080 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:34.669529 (+ 2449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:35:34.669539 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:35:34.669558 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:35:34.669593 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:35:34.671489 (+ 1896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:35:34.671908 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:35:34.671914 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:35:34.671923 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:35:34.672063 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:35:34.672306 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:35:34.672309 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:35:34.672310 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:35:34.672815 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:35:34.673000 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0929 10:35:34.673268 (+ 268us) server_negotiation.cc:299] Negotiation successful
0929 10:35:34.673552 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":119,"threads_started":1}
I20260929 10:35:34.674875 27975 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a751487001594e6293a13894cf876e50: No bootstrap required, opened a new log
I20260929 10:35:34.678025 27975 raft_consensus.cc:348] T 00000000000000000000000000000000 P a751487001594e6293a13894cf876e50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a751487001594e6293a13894cf876e50" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 33739 } }
I20260929 10:35:34.678232 27975 raft_consensus.cc:374] T 00000000000000000000000000000000 P a751487001594e6293a13894cf876e50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:35:34.678267 27975 raft_consensus.cc:729] T 00000000000000000000000000000000 P a751487001594e6293a13894cf876e50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a751487001594e6293a13894cf876e50, State: Initialized, Role: FOLLOWER
I20260929 10:35:34.678697 27975 consensus_queue.cc:260] T 00000000000000000000000000000000 P a751487001594e6293a13894cf876e50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a751487001594e6293a13894cf876e50" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 33739 } }
I20260929 10:35:34.678828 27975 raft_consensus.cc:388] T 00000000000000000000000000000000 P a751487001594e6293a13894cf876e50 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:35:34.678872 27975 raft_consensus.cc:482] T 00000000000000000000000000000000 P a751487001594e6293a13894cf876e50 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:35:34.678942 27975 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a751487001594e6293a13894cf876e50 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:35:34.679755 27975 raft_consensus.cc:504] T 00000000000000000000000000000000 P a751487001594e6293a13894cf876e50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a751487001594e6293a13894cf876e50" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 33739 } }
I20260929 10:35:34.680070 27975 leader_election.cc:302] T 00000000000000000000000000000000 P a751487001594e6293a13894cf876e50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a751487001594e6293a13894cf876e50; no voters:
I20260929 10:35:34.680311 27975 leader_election.cc:288] T 00000000000000000000000000000000 P a751487001594e6293a13894cf876e50 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:35:34.680403 27980 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a751487001594e6293a13894cf876e50 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:35:34.680686 27980 raft_consensus.cc:686] T 00000000000000000000000000000000 P a751487001594e6293a13894cf876e50 [term 1 LEADER]: Becoming Leader. State: Replica: a751487001594e6293a13894cf876e50, State: Running, Role: LEADER
I20260929 10:35:34.681038 27980 consensus_queue.cc:237] T 00000000000000000000000000000000 P a751487001594e6293a13894cf876e50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a751487001594e6293a13894cf876e50" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 33739 } }
I20260929 10:35:34.681531 27975 sys_catalog.cc:564] T 00000000000000000000000000000000 P a751487001594e6293a13894cf876e50 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:35:34.682763 27982 sys_catalog.cc:455] T 00000000000000000000000000000000 P a751487001594e6293a13894cf876e50 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a751487001594e6293a13894cf876e50. Latest consensus state: current_term: 1 leader_uuid: "a751487001594e6293a13894cf876e50" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a751487001594e6293a13894cf876e50" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 33739 } } }
I20260929 10:35:34.682891 27982 sys_catalog.cc:458] T 00000000000000000000000000000000 P a751487001594e6293a13894cf876e50 [sys.catalog]: This master's current role is: LEADER
I20260929 10:35:34.683256 27989 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:35:34.683290 27981 sys_catalog.cc:455] T 00000000000000000000000000000000 P a751487001594e6293a13894cf876e50 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a751487001594e6293a13894cf876e50" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a751487001594e6293a13894cf876e50" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 33739 } } }
I20260929 10:35:34.683439 27981 sys_catalog.cc:458] T 00000000000000000000000000000000 P a751487001594e6293a13894cf876e50 [sys.catalog]: This master's current role is: LEADER
I20260929 10:35:34.686096 27989 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:35:34.686523 27975 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260929 10:35:34.695349 27989 catalog_manager.cc:1347] Generated new cluster ID: eaae6427353f4f1fa8f6aeaaed9b522c
I20260929 10:35:34.695436 27989 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:35:34.709301 27989 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:35:34.710307 27989 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:35:34.725389 27989 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a751487001594e6293a13894cf876e50: Generated new TSK 0
I20260929 10:35:34.726972 27989 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.82.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab.
Entry for principal kudu/127.15.82.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab.
I20260929 10:35:35.445996 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.82.84:34645/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.65:0
--local_ip_for_outbound_sockets=127.15.82.65
--webserver_interface=127.15.82.65
--webserver_port=0
--tserver_master_addrs=127.15.82.126:33739
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab
--principal=kudu/127.15.82.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:44449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:35:35.566507 28020 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:35:35.566780 28020 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:35:35.566847 28020 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:35:35.571267 28020 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:35:35.571352 28020 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:35:35.571455 28020 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.65
I20260929 10:35:35.577077 28020 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:44449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.15.82.84:34645/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab
--principal=kudu/127.15.82.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.82.65
--webserver_port=0
--tserver_master_addrs=127.15.82.126:33739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28020
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.65
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:35:35.578187 28020 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:35:35.579308 28020 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:35:35.589911 28026 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:35:35.589924 28028 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:35:35.590083 28025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:35:35.590579 28020 server_base.cc:1034] running on GCE node
I20260929 10:35:35.591099 28020 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:35:35.591780 28020 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:35:35.592962 28020 hybrid_clock.cc:648] HybridClock initialized: now 1790678135592923 us; error 53 us; skew 500 ppm
Sep 29 10:35:35 dist-test-slave-vg89 krb5kdc[27463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678135, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:35:35.596549 28020 init.cc:377] Logged in from keytab as kudu/127.15.82.65@KRBTEST.COM (short username kudu)
I20260929 10:35:35.597553 28020 webserver.cc:460] Webserver started at http://127.15.82.65:42123/ using document root <none> and password file <none>
I20260929 10:35:35.598193 28020 fs_manager.cc:359] Metadata directory not provided
I20260929 10:35:35.598289 28020 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:35:35.598564 28020 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 10:35:35 dist-test-slave-vg89 krb5kdc[27463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678135, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.65@KRBTEST.COM for HTTP/127.15.82.84@KRBTEST.COM
I20260929 10:35:35.615221 28020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-0/data/instance:
uuid: "814e446a7d1e44589801e2242186d5d0"
format_stamp: "Formatted at 2026-09-29 10:35:35 on dist-test-slave-vg89"
server_key: "9db64d9bc967cd08fe0dc896530bb6b3"
server_key_iv: "d01ad1f292f519dbedf32acd30272e21"
server_key_version: "kuduclusterkey@0"
I20260929 10:35:35.616051 28020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-0/wal/instance:
uuid: "814e446a7d1e44589801e2242186d5d0"
format_stamp: "Formatted at 2026-09-29 10:35:35 on dist-test-slave-vg89"
server_key: "9db64d9bc967cd08fe0dc896530bb6b3"
server_key_iv: "d01ad1f292f519dbedf32acd30272e21"
server_key_version: "kuduclusterkey@0"
I20260929 10:35:35.621129 28020 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260929 10:35:35.652256 28037 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:35:35.654083 28020 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260929 10:35:35.654223 28020 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-0/wal
uuid: "814e446a7d1e44589801e2242186d5d0"
format_stamp: "Formatted at 2026-09-29 10:35:35 on dist-test-slave-vg89"
server_key: "9db64d9bc967cd08fe0dc896530bb6b3"
server_key_iv: "d01ad1f292f519dbedf32acd30272e21"
server_key_version: "kuduclusterkey@0"
I20260929 10:35:35.654364 28020 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:35:35.686164 28020 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:35:35.699283 28020 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:35:35.699517 28020 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:35:35.700295 28020 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:35:35.701406 28020 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:35:35.701524 28020 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:35:35.701653 28020 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:35:35.701731 28020 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:35:35.714341 28020 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.65:42219
I20260929 10:35:35.714365 28152 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.65:42219 every 8 connection(s)
I20260929 10:35:35.715436 28020 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
I20260929 10:35:35.715715 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 28020
I20260929 10:35:35.715847 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-0/wal/instance
I20260929 10:35:35.716198 15689 external_mini_cluster.cc:1428] Setting key b79c67b1e34de722d427e2bc79219c99
WARNING: no policy specified for kudu/127.15.82.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.66@KRBTEST.COM" created.
Sep 29 10:35:35 dist-test-slave-vg89 krb5kdc[27463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678135, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.65@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:35:35.767552 28158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:35.742497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.65:41963 (local address 127.15.82.126:33739)
0929 10:35:35.742761 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0929 10:35:35.742766 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:35:35.751303 (+ 8537us) server_negotiation.cc:407] Connection header received
0929 10:35:35.751363 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:35:35.751366 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:35:35.751425 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:35:35.751528 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:35:35.754202 (+ 2674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:35.755144 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:35.757050 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:35.757453 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:35.757548 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:35.762506 (+ 4958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:35:35.762511 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:35:35.762513 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:35:35.762543 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:35:35.764600 (+ 2057us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:35:35.765307 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:35:35.765311 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:35:35.765313 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:35:35.765406 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:35:35.765847 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:35:35.765850 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:35:35.765852 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:35:35.766115 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:35:35.766189 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0929 10:35:35.767140 (+ 951us) server_negotiation.cc:299] Negotiation successful
0929 10:35:35.767278 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":90,"threads_started":1}
I20260929 10:35:35.769757 28157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:35.741309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:33739 (local address 127.15.82.65:41963)
0929 10:35:35.749742 (+ 8433us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:35.749814 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0929 10:35:35.750962 (+ 1148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:35:35.751855 (+ 893us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:35:35.751865 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:35:35.752241 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:35:35.754014 (+ 1773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:35.754035 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:35.755325 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:35.755331 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:35.756899 (+ 1568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:35.756910 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:35.757785 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:35.757791 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:35.758017 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:35.759169 (+ 1152us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:35:35.759190 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:35:35.762230 (+ 3040us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:35:35.764734 (+ 2504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:35:35.764740 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:35:35.764757 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:35:35.765106 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:35:35.765515 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:35:35.765518 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:35:35.765520 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:35:35.765691 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:35:35.766272 (+ 581us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:35:35.766278 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:35:35.766762 (+ 484us) client_negotiation.cc:769] Sending connection context
0929 10:35:35.768278 (+ 1516us) client_negotiation.cc:240] Negotiation successful
0929 10:35:35.768643 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8160,"thread_start_us":167,"threads_started":1}
I20260929 10:35:35.771667 28153 heartbeater.cc:347] Connected to a master server at 127.15.82.126:33739
I20260929 10:35:35.771905 28153 heartbeater.cc:464] Registering TS with master...
I20260929 10:35:35.772476 28153 heartbeater.cc:511] Master 127.15.82.126:33739 requested a full tablet report, sending...
I20260929 10:35:35.774231 27939 ts_manager.cc:194] Registered new tserver with Master: 814e446a7d1e44589801e2242186d5d0 (127.15.82.65:42219)
I20260929 10:35:35.775823 27939 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.82.65@KRBTEST.COM'} at 127.15.82.65:41963
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab.
Entry for principal kudu/127.15.82.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab.
I20260929 10:35:35.782163 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.82.84:34645/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.66:0
--local_ip_for_outbound_sockets=127.15.82.66
--webserver_interface=127.15.82.66
--webserver_port=0
--tserver_master_addrs=127.15.82.126:33739
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab
--principal=kudu/127.15.82.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:44449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 10:35:35 dist-test-slave-vg89 krb5kdc[27463](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790678135, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260929 10:35:35.945050 28159 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:35:35.945411 28159 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:35:35.945477 28159 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:35:35.951611 28159 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:35:35.951691 28159 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:35:35.951777 28159 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.66
I20260929 10:35:35.959556 28159 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:44449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.15.82.84:34645/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab
--principal=kudu/127.15.82.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.82.66
--webserver_port=0
--tserver_master_addrs=127.15.82.126:33739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28159
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.66
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:35:35.961123 28159 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:35:35.962394 28159 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:35:35.972160 28168 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:35:35.972153 28165 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:35:35.972595 28159 server_base.cc:1034] running on GCE node
W20260929 10:35:35.972867 28166 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:35:35.973359 28159 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:35:35.974118 28159 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:35:35.975344 28159 hybrid_clock.cc:648] HybridClock initialized: now 1790678135975275 us; error 85 us; skew 500 ppm
Sep 29 10:35:35 dist-test-slave-vg89 krb5kdc[27463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678135, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:35:35.979001 28159 init.cc:377] Logged in from keytab as kudu/127.15.82.66@KRBTEST.COM (short username kudu)
I20260929 10:35:35.980072 28159 webserver.cc:460] Webserver started at http://127.15.82.66:35797/ using document root <none> and password file <none>
I20260929 10:35:35.980793 28159 fs_manager.cc:359] Metadata directory not provided
I20260929 10:35:35.980873 28159 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:35:35.981271 28159 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 10:35:35 dist-test-slave-vg89 krb5kdc[27463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678135, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.66@KRBTEST.COM for HTTP/127.15.82.84@KRBTEST.COM
I20260929 10:35:36.000412 28159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-1/data/instance:
uuid: "492cde4f6775496bbad5555ed4d806f4"
format_stamp: "Formatted at 2026-09-29 10:35:35 on dist-test-slave-vg89"
server_key: "00d75bdf3b5efb24ec66415ea5d5f876"
server_key_iv: "4006edaf0250f72db850a6eb027da3f1"
server_key_version: "kuduclusterkey@0"
I20260929 10:35:36.004396 28159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-1/wal/instance:
uuid: "492cde4f6775496bbad5555ed4d806f4"
format_stamp: "Formatted at 2026-09-29 10:35:35 on dist-test-slave-vg89"
server_key: "00d75bdf3b5efb24ec66415ea5d5f876"
server_key_iv: "4006edaf0250f72db850a6eb027da3f1"
server_key_version: "kuduclusterkey@0"
I20260929 10:35:36.009737 28159 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260929 10:35:36.037578 28175 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:35:36.039530 28159 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.004s sys 0.000s
I20260929 10:35:36.039695 28159 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-1/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-1/wal
uuid: "492cde4f6775496bbad5555ed4d806f4"
format_stamp: "Formatted at 2026-09-29 10:35:35 on dist-test-slave-vg89"
server_key: "00d75bdf3b5efb24ec66415ea5d5f876"
server_key_iv: "4006edaf0250f72db850a6eb027da3f1"
server_key_version: "kuduclusterkey@0"
I20260929 10:35:36.039853 28159 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:35:36.080014 28159 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:35:36.084005 28159 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:35:36.084203 28159 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:35:36.084919 28159 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:35:36.085913 28159 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:35:36.085991 28159 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:35:36.086053 28159 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:35:36.086086 28159 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:35:36.096760 28159 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.66:42351
I20260929 10:35:36.097455 28288 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.66:42351 every 8 connection(s)
I20260929 10:35:36.097842 28159 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
I20260929 10:35:36.099483 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 28159
I20260929 10:35:36.099615 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-1/wal/instance
I20260929 10:35:36.099931 15689 external_mini_cluster.cc:1428] Setting key 2afd71f51174d10ec64c6b748fffd25c
Sep 29 10:35:36 dist-test-slave-vg89 krb5kdc[27463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678135, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.66@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.82.67@KRBTEST.COM; defaulting to no policy
I20260929 10:35:36.115783 28158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:36.101427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:52335 (local address 127.15.82.126:33739)
0929 10:35:36.101547 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0929 10:35:36.101551 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:35:36.102290 (+ 739us) server_negotiation.cc:407] Connection header received
0929 10:35:36.103722 (+ 1432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:35:36.103725 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:35:36.103774 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:35:36.103848 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:35:36.105391 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:36.105941 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:36.107780 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:36.108226 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:36.108315 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:36.111499 (+ 3184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:35:36.111504 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:35:36.111507 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:35:36.111537 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:35:36.113422 (+ 1885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:35:36.114094 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:35:36.114098 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:35:36.114100 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:35:36.114181 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:35:36.114604 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:35:36.114607 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:35:36.114609 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:35:36.114789 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:35:36.114893 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0929 10:35:36.115475 (+ 582us) server_negotiation.cc:299] Negotiation successful
0929 10:35:36.115607 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.67@KRBTEST.COM" created.
I20260929 10:35:36.117089 28291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:36.100454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:33739 (local address 127.15.82.66:52335)
0929 10:35:36.102149 (+ 1695us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:36.102202 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0929 10:35:36.103498 (+ 1296us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:35:36.104161 (+ 663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:35:36.104172 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:35:36.104562 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:35:36.105250 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:36.105267 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:36.106067 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:36.106074 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:36.107630 (+ 1556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:36.107644 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:36.108534 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:36.108541 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:36.108712 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:36.109297 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:35:36.109322 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:35:36.111286 (+ 1964us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:35:36.113623 (+ 2337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:35:36.113632 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:35:36.113650 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:35:36.113981 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:35:36.114281 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:35:36.114284 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:35:36.114286 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:35:36.114482 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:35:36.114890 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:35:36.114897 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:35:36.115236 (+ 339us) client_negotiation.cc:769] Sending connection context
0929 10:35:36.115421 (+ 185us) client_negotiation.cc:240] Negotiation successful
0929 10:35:36.115747 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1465,"thread_start_us":129,"threads_started":1}
I20260929 10:35:36.118381 28289 heartbeater.cc:347] Connected to a master server at 127.15.82.126:33739
I20260929 10:35:36.118691 28289 heartbeater.cc:464] Registering TS with master...
I20260929 10:35:36.119277 28289 heartbeater.cc:511] Master 127.15.82.126:33739 requested a full tablet report, sending...
I20260929 10:35:36.120504 27939 ts_manager.cc:194] Registered new tserver with Master: 492cde4f6775496bbad5555ed4d806f4 (127.15.82.66:42351)
I20260929 10:35:36.121186 27939 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.82.66@KRBTEST.COM'} at 127.15.82.66:52335
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab.
Entry for principal kudu/127.15.82.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab.
I20260929 10:35:36.138355 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.82.84:34645/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.67:0
--local_ip_for_outbound_sockets=127.15.82.67
--webserver_interface=127.15.82.67
--webserver_port=0
--tserver_master_addrs=127.15.82.126:33739
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab
--principal=kudu/127.15.82.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:44449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:35:36.278127 28294 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:35:36.278393 28294 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:35:36.278501 28294 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:35:36.283203 28294 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:35:36.283267 28294 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:35:36.283345 28294 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.67
I20260929 10:35:36.291421 28294 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:44449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.15.82.84:34645/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab
--principal=kudu/127.15.82.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.82.67
--webserver_port=0
--tserver_master_addrs=127.15.82.126:33739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28294
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.67
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:35:36.292860 28294 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:35:36.294379 28294 file_cache.cc:493] Constructed file cache file cache with capacity 419430
Sep 29 10:35:36 dist-test-slave-vg89 krb5kdc[27463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678135, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.126@KRBTEST.COM for HTTP/127.15.82.84@KRBTEST.COM
W20260929 10:35:36.308349 28300 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:35:36.308830 28302 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:35:36.309062 28299 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:35:36.315107 28294 server_base.cc:1034] running on GCE node
I20260929 10:35:36.315726 28294 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:35:36.316608 28294 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:35:36.317801 28294 hybrid_clock.cc:648] HybridClock initialized: now 1790678136317729 us; error 86 us; skew 500 ppm
Sep 29 10:35:36 dist-test-slave-vg89 krb5kdc[27463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678136, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:35:36.325488 28294 init.cc:377] Logged in from keytab as kudu/127.15.82.67@KRBTEST.COM (short username kudu)
I20260929 10:35:36.326699 28294 webserver.cc:460] Webserver started at http://127.15.82.67:38957/ using document root <none> and password file <none>
I20260929 10:35:36.327451 28294 fs_manager.cc:359] Metadata directory not provided
I20260929 10:35:36.327543 28294 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:35:36.327780 28294 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 10:35:36 dist-test-slave-vg89 krb5kdc[27463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678136, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.67@KRBTEST.COM for HTTP/127.15.82.84@KRBTEST.COM
I20260929 10:35:36.352556 28294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-2/data/instance:
uuid: "76b89a15d6834c66b8f4932be823bd78"
format_stamp: "Formatted at 2026-09-29 10:35:36 on dist-test-slave-vg89"
server_key: "48739dbe4524e9f58c7cf50433e090e6"
server_key_iv: "07cd96e0034d4bef871600a4dfa5ca3f"
server_key_version: "kuduclusterkey@0"
I20260929 10:35:36.353336 28294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-2/wal/instance:
uuid: "76b89a15d6834c66b8f4932be823bd78"
format_stamp: "Formatted at 2026-09-29 10:35:36 on dist-test-slave-vg89"
server_key: "48739dbe4524e9f58c7cf50433e090e6"
server_key_iv: "07cd96e0034d4bef871600a4dfa5ca3f"
server_key_version: "kuduclusterkey@0"
I20260929 10:35:36.357915 28294 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260929 10:35:36.380918 28309 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:35:36.382786 28294 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 10:35:36.382910 28294 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-2/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-2/wal
uuid: "76b89a15d6834c66b8f4932be823bd78"
format_stamp: "Formatted at 2026-09-29 10:35:36 on dist-test-slave-vg89"
server_key: "48739dbe4524e9f58c7cf50433e090e6"
server_key_iv: "07cd96e0034d4bef871600a4dfa5ca3f"
server_key_version: "kuduclusterkey@0"
I20260929 10:35:36.383091 28294 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:35:36.397068 28294 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:35:36.400825 28294 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:35:36.401039 28294 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:35:36.401742 28294 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:35:36.402833 28294 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:35:36.402901 28294 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:35:36.402966 28294 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:35:36.402994 28294 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:35:36.413945 28294 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.67:33605
I20260929 10:35:36.414183 28422 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.67:33605 every 8 connection(s)
I20260929 10:35:36.415717 28294 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
I20260929 10:35:36.424393 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 28294
I20260929 10:35:36.424510 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-2/wal/instance
I20260929 10:35:36.424824 15689 external_mini_cluster.cc:1428] Setting key 6259b7946f0ec3dfa656df2e19cabacc
Sep 29 10:35:36 dist-test-slave-vg89 krb5kdc[27463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678136, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.67@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:35:36.433586 28158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:36.419680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.67:46095 (local address 127.15.82.126:33739)
0929 10:35:36.419806 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0929 10:35:36.419811 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:35:36.419831 (+ 20us) server_negotiation.cc:407] Connection header received
0929 10:35:36.420481 (+ 650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:35:36.420484 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:35:36.420536 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:35:36.420633 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:35:36.422027 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:36.422858 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:36.425568 (+ 2710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:36.425892 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:36.425962 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:36.429109 (+ 3147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:35:36.429113 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:35:36.429116 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:35:36.429144 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:35:36.430991 (+ 1847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:35:36.431642 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:35:36.431646 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:35:36.431650 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:35:36.431755 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:35:36.432187 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:35:36.432191 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:35:36.432194 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:35:36.432438 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:35:36.432557 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0929 10:35:36.433237 (+ 680us) server_negotiation.cc:299] Negotiation successful
0929 10:35:36.433378 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260929 10:35:36.434725 28425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:36.418485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:33739 (local address 127.15.82.67:46095)
0929 10:35:36.419108 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:36.419170 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0929 10:35:36.420228 (+ 1058us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:35:36.420837 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:35:36.420847 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:35:36.421215 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:35:36.421869 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:36.421888 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:36.423008 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:36.423011 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:36.425412 (+ 2401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:36.425423 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:36.426166 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:36.426169 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:36.426360 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:36.427114 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:35:36.427134 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:35:36.428878 (+ 1744us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:35:36.431130 (+ 2252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:35:36.431137 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:35:36.431155 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:35:36.431523 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:35:36.431862 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:35:36.431865 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:35:36.431867 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:35:36.432073 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:35:36.432565 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:35:36.432573 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:35:36.432943 (+ 370us) client_negotiation.cc:769] Sending connection context
0929 10:35:36.433168 (+ 225us) client_negotiation.cc:240] Negotiation successful
0929 10:35:36.433563 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":158,"threads_started":1}
I20260929 10:35:36.436054 28423 heartbeater.cc:347] Connected to a master server at 127.15.82.126:33739
I20260929 10:35:36.436357 28423 heartbeater.cc:464] Registering TS with master...
I20260929 10:35:36.436901 28423 heartbeater.cc:511] Master 127.15.82.126:33739 requested a full tablet report, sending...
I20260929 10:35:36.438027 27939 ts_manager.cc:194] Registered new tserver with Master: 76b89a15d6834c66b8f4932be823bd78 (127.15.82.67:33605)
I20260929 10:35:36.438787 27939 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.82.67@KRBTEST.COM'} at 127.15.82.67:46095
I20260929 10:35:36.439147 15689 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 10:35:36.439997 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 28020
I20260929 10:35:36.447110 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 28159
I20260929 10:35:36.454258 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 28294
I20260929 10:35:36.461555 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 27907
I20260929 10:35:36.467132 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:33739
--webserver_interface=127.15.82.126
--webserver_port=36465
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab
--principal=kudu/127.15.82.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:44449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.82.126:33739
--ranger_config_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.82.84:34645/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.15.82.84:34645/kms with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:35:36.578568 28426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:35:36.578855 28426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:35:36.578895 28426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:35:36.583124 28426 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:35:36.583181 28426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:35:36.583200 28426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:35:36.583216 28426 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:35:36.583249 28426 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:35:36.588732 28426 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:44449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.15.82.84:34645/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.15.82.126:33739
--ranger_config_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab
--principal=kudu/127.15.82.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:33739
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.82.126
--webserver_port=36465
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28426
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:35:36.589779 28426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:35:36.590862 28426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:35:36.599253 28434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:35:36.599253 28431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:35:36.599270 28432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:35:36.599465 28426 server_base.cc:1034] running on GCE node
I20260929 10:35:36.600333 28426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:35:36.601361 28426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:35:36.602483 28426 hybrid_clock.cc:648] HybridClock initialized: now 1790678136602439 us; error 45 us; skew 500 ppm
Sep 29 10:35:36 dist-test-slave-vg89 krb5kdc[27463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678136, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:35:36.605291 28426 init.cc:377] Logged in from keytab as kudu/127.15.82.126@KRBTEST.COM (short username kudu)
I20260929 10:35:36.606181 28426 webserver.cc:460] Webserver started at http://127.15.82.126:36465/ using document root <none> and password file <none>
I20260929 10:35:36.606776 28426 fs_manager.cc:359] Metadata directory not provided
I20260929 10:35:36.606865 28426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 29 10:35:36 dist-test-slave-vg89 krb5kdc[27463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678136, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.126@KRBTEST.COM for HTTP/127.15.82.84@KRBTEST.COM
I20260929 10:35:36.667112 28426 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260929 10:35:36.669811 28442 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:35:36.671406 28426 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 10:35:36.671561 28426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/master-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/master-0/wal
uuid: "a751487001594e6293a13894cf876e50"
format_stamp: "Formatted at 2026-09-29 10:35:34 on dist-test-slave-vg89"
server_key: "3a5bc85ba5a60891e232a5e51a2ac65e"
server_key_iv: "c2504553238db51241ea80876b271619"
server_key_version: "kuduclusterkey@0"
I20260929 10:35:36.672260 28426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:35:36.689303 28426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:35:36.692229 28426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:35:36.692643 28426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:35:36.699776 28426 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.126:33739
I20260929 10:35:36.699797 28494 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.126:33739 every 8 connection(s)
I20260929 10:35:36.700819 28426 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
I20260929 10:35:36.704839 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 28426
I20260929 10:35:36.705053 28495 sys_catalog.cc:263] Verifying existing consensus state
I20260929 10:35:36.705339 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.82.84:34645/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.65:42219
--local_ip_for_outbound_sockets=127.15.82.65
--tserver_master_addrs=127.15.82.126:33739
--webserver_port=42123
--webserver_interface=127.15.82.65
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab
--principal=kudu/127.15.82.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:44449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.15.82.84:34645/kms with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 10:35:36.706772 28495 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a751487001594e6293a13894cf876e50: Bootstrap starting.
I20260929 10:35:36.722784 28495 log.cc:824] T 00000000000000000000000000000000 P a751487001594e6293a13894cf876e50: Log is configured to *not* fsync() on all Append() calls
I20260929 10:35:36.729111 28495 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a751487001594e6293a13894cf876e50: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 10:35:36.729537 28495 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a751487001594e6293a13894cf876e50: Bootstrap complete.
I20260929 10:35:36.733641 28495 raft_consensus.cc:348] T 00000000000000000000000000000000 P a751487001594e6293a13894cf876e50 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a751487001594e6293a13894cf876e50" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 33739 } }
I20260929 10:35:36.734037 28495 raft_consensus.cc:729] T 00000000000000000000000000000000 P a751487001594e6293a13894cf876e50 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a751487001594e6293a13894cf876e50, State: Initialized, Role: FOLLOWER
I20260929 10:35:36.734547 28495 consensus_queue.cc:260] T 00000000000000000000000000000000 P a751487001594e6293a13894cf876e50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a751487001594e6293a13894cf876e50" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 33739 } }
I20260929 10:35:36.734680 28495 raft_consensus.cc:388] T 00000000000000000000000000000000 P a751487001594e6293a13894cf876e50 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:35:36.734760 28495 raft_consensus.cc:482] T 00000000000000000000000000000000 P a751487001594e6293a13894cf876e50 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:35:36.735054 28495 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a751487001594e6293a13894cf876e50 [term 1 FOLLOWER]: Advancing to term 2
I20260929 10:35:36.737003 28495 raft_consensus.cc:504] T 00000000000000000000000000000000 P a751487001594e6293a13894cf876e50 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a751487001594e6293a13894cf876e50" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 33739 } }
I20260929 10:35:36.737334 28495 leader_election.cc:302] T 00000000000000000000000000000000 P a751487001594e6293a13894cf876e50 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a751487001594e6293a13894cf876e50; no voters:
I20260929 10:35:36.737686 28495 leader_election.cc:288] T 00000000000000000000000000000000 P a751487001594e6293a13894cf876e50 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260929 10:35:36.737775 28499 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a751487001594e6293a13894cf876e50 [term 2 FOLLOWER]: Leader election won for term 2
I20260929 10:35:36.738058 28499 raft_consensus.cc:686] T 00000000000000000000000000000000 P a751487001594e6293a13894cf876e50 [term 2 LEADER]: Becoming Leader. State: Replica: a751487001594e6293a13894cf876e50, State: Running, Role: LEADER
I20260929 10:35:36.738363 28499 consensus_queue.cc:237] T 00000000000000000000000000000000 P a751487001594e6293a13894cf876e50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a751487001594e6293a13894cf876e50" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 33739 } }
I20260929 10:35:36.739044 28495 sys_catalog.cc:564] T 00000000000000000000000000000000 P a751487001594e6293a13894cf876e50 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:35:36.739807 28501 sys_catalog.cc:455] T 00000000000000000000000000000000 P a751487001594e6293a13894cf876e50 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "a751487001594e6293a13894cf876e50" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a751487001594e6293a13894cf876e50" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 33739 } } }
I20260929 10:35:36.739969 28501 sys_catalog.cc:458] T 00000000000000000000000000000000 P a751487001594e6293a13894cf876e50 [sys.catalog]: This master's current role is: LEADER
I20260929 10:35:36.740329 28500 sys_catalog.cc:455] T 00000000000000000000000000000000 P a751487001594e6293a13894cf876e50 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a751487001594e6293a13894cf876e50. Latest consensus state: current_term: 2 leader_uuid: "a751487001594e6293a13894cf876e50" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a751487001594e6293a13894cf876e50" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 33739 } } }
I20260929 10:35:36.740479 28500 sys_catalog.cc:458] T 00000000000000000000000000000000 P a751487001594e6293a13894cf876e50 [sys.catalog]: This master's current role is: LEADER
I20260929 10:35:36.740761 28506 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:35:36.743953 28506 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:35:36.744511 28495 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260929 10:35:36.744676 28506 catalog_manager.cc:1259] Loaded cluster ID: eaae6427353f4f1fa8f6aeaaed9b522c
I20260929 10:35:36.744764 28506 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:35:36.747172 28506 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:35:36.747782 28506 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a751487001594e6293a13894cf876e50: Loaded TSK: 0
I20260929 10:35:36.748988 28506 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260929 10:35:36.844066 28497 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:35:36.844364 28497 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:35:36.844444 28497 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:35:36.849494 28497 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:35:36.849560 28497 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:35:36.849643 28497 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.65
I20260929 10:35:36.855391 28497 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:44449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.15.82.84:34645/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab
--principal=kudu/127.15.82.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.65:42219
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.82.65
--webserver_port=42123
--tserver_master_addrs=127.15.82.126:33739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28497
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.65
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:35:36.856492 28497 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:35:36.857822 28497 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:35:36.869622 28535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:35:36.869625 28533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:35:36.869944 28532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:35:36.869963 28497 server_base.cc:1034] running on GCE node
I20260929 10:35:36.870468 28497 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:35:36.871291 28497 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:35:36.872476 28497 hybrid_clock.cc:648] HybridClock initialized: now 1790678136872445 us; error 49 us; skew 500 ppm
Sep 29 10:35:36 dist-test-slave-vg89 krb5kdc[27463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678136, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:35:36.876133 28497 init.cc:377] Logged in from keytab as kudu/127.15.82.65@KRBTEST.COM (short username kudu)
I20260929 10:35:36.877357 28497 webserver.cc:460] Webserver started at http://127.15.82.65:42123/ using document root <none> and password file <none>
I20260929 10:35:36.878182 28497 fs_manager.cc:359] Metadata directory not provided
I20260929 10:35:36.878296 28497 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 29 10:35:36 dist-test-slave-vg89 krb5kdc[27463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678136, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.65@KRBTEST.COM for HTTP/127.15.82.84@KRBTEST.COM
I20260929 10:35:36.900309 28497 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.002s sys 0.000s
I20260929 10:35:36.903693 28543 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:35:36.905395 28497 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260929 10:35:36.905532 28497 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-0/wal
uuid: "814e446a7d1e44589801e2242186d5d0"
format_stamp: "Formatted at 2026-09-29 10:35:35 on dist-test-slave-vg89"
server_key: "9db64d9bc967cd08fe0dc896530bb6b3"
server_key_iv: "d01ad1f292f519dbedf32acd30272e21"
server_key_version: "kuduclusterkey@0"
I20260929 10:35:36.906270 28497 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:35:36.920794 28497 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:35:36.924383 28497 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:35:36.924598 28497 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:35:36.925323 28497 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:35:36.926332 28497 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:35:36.926421 28497 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:35:36.926486 28497 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:35:36.926517 28497 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:35:36.937553 28497 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.65:42219
I20260929 10:35:36.937654 28656 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.65:42219 every 8 connection(s)
I20260929 10:35:36.938884 28497 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
I20260929 10:35:36.946734 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 28497
I20260929 10:35:36.947253 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.82.84:34645/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.66:42351
--local_ip_for_outbound_sockets=127.15.82.66
--tserver_master_addrs=127.15.82.126:33739
--webserver_port=35797
--webserver_interface=127.15.82.66
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab
--principal=kudu/127.15.82.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:44449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.15.82.84:34645/kms with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 10:35:36 dist-test-slave-vg89 krb5kdc[27463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678136, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.65@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:35:36.964017 28660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:36.941431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:33739 (local address 127.15.82.65:46611)
0929 10:35:36.942543 (+ 1112us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:36.942612 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0929 10:35:36.943540 (+ 928us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:35:36.944928 (+ 1388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:35:36.944936 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:35:36.945276 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:35:36.945915 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:36.945936 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:36.951200 (+ 5264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:36.951206 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:36.952629 (+ 1423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:36.952640 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:36.954068 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:36.954075 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:36.954317 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:36.955133 (+ 816us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:35:36.955159 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:35:36.957087 (+ 1928us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:35:36.960015 (+ 2928us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:35:36.960024 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:35:36.960044 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:35:36.960424 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:35:36.960878 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:35:36.960884 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:35:36.960889 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:35:36.961098 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:35:36.962003 (+ 905us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:35:36.962012 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:35:36.962356 (+ 344us) client_negotiation.cc:769] Sending connection context
0929 10:35:36.962625 (+ 269us) client_negotiation.cc:240] Negotiation successful
0929 10:35:36.963008 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":876,"thread_start_us":135,"threads_started":1}
I20260929 10:35:36.965754 28659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:36.941273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.65:46611 (local address 127.15.82.126:33739)
0929 10:35:36.941822 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0929 10:35:36.941845 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0929 10:35:36.942748 (+ 903us) server_negotiation.cc:407] Connection header received
0929 10:35:36.943996 (+ 1248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:35:36.944010 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:35:36.944474 (+ 464us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:35:36.944855 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:35:36.949408 (+ 4553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:36.951013 (+ 1605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:36.952759 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:36.953624 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:36.953811 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:36.957345 (+ 3534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:35:36.957355 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:35:36.957377 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:35:36.957417 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:35:36.959846 (+ 2429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:35:36.960582 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:35:36.960592 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:35:36.960602 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:35:36.960755 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:35:36.961226 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:35:36.961232 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:35:36.961236 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:35:36.961781 (+ 545us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:35:36.961939 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0929 10:35:36.964578 (+ 2639us) server_negotiation.cc:299] Negotiation successful
0929 10:35:36.964868 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":131,"threads_started":1}
I20260929 10:35:36.968943 28657 heartbeater.cc:347] Connected to a master server at 127.15.82.126:33739
I20260929 10:35:36.969169 28657 heartbeater.cc:464] Registering TS with master...
I20260929 10:35:36.969698 28657 heartbeater.cc:511] Master 127.15.82.126:33739 requested a full tablet report, sending...
W20260929 10:35:36.972035 28657 heartbeater.cc:644] Failed to heartbeat to 127.15.82.126:33739 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260929 10:35:36.972149 28657 heartbeater.cc:464] Registering TS with master...
I20260929 10:35:36.972301 28657 heartbeater.cc:511] Master 127.15.82.126:33739 requested a full tablet report, sending...
I20260929 10:35:36.973134 28657 heartbeater.cc:464] Registering TS with master...
I20260929 10:35:36.973278 28657 heartbeater.cc:511] Master 127.15.82.126:33739 requested a full tablet report, sending...
W20260929 10:35:36.974042 28657 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 10:35:37.100385 28661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:35:37.100740 28661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:35:37.100822 28661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:35:37.106705 28661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:35:37.106796 28661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:35:37.106892 28661 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.66
I20260929 10:35:37.113853 28661 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:44449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.15.82.84:34645/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab
--principal=kudu/127.15.82.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.66:42351
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.82.66
--webserver_port=35797
--tserver_master_addrs=127.15.82.126:33739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28661
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.66
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:35:37.115200 28661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:35:37.116325 28661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:35:37.128012 28666 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:35:37.128533 28667 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:35:37.128818 28661 server_base.cc:1034] running on GCE node
W20260929 10:35:37.128036 28669 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:35:37.129505 28661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:35:37.130247 28661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:35:37.131419 28661 hybrid_clock.cc:648] HybridClock initialized: now 1790678137131383 us; error 54 us; skew 500 ppm
Sep 29 10:35:37 dist-test-slave-vg89 krb5kdc[27463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678137, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:35:37.135126 28661 init.cc:377] Logged in from keytab as kudu/127.15.82.66@KRBTEST.COM (short username kudu)
I20260929 10:35:37.136200 28661 webserver.cc:460] Webserver started at http://127.15.82.66:35797/ using document root <none> and password file <none>
I20260929 10:35:37.136880 28661 fs_manager.cc:359] Metadata directory not provided
I20260929 10:35:37.136993 28661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 29 10:35:37 dist-test-slave-vg89 krb5kdc[27463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678137, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.66@KRBTEST.COM for HTTP/127.15.82.84@KRBTEST.COM
I20260929 10:35:37.160657 28661 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260929 10:35:37.163273 28677 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:35:37.164615 28661 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 10:35:37.164741 28661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-1/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-1/wal
uuid: "492cde4f6775496bbad5555ed4d806f4"
format_stamp: "Formatted at 2026-09-29 10:35:35 on dist-test-slave-vg89"
server_key: "00d75bdf3b5efb24ec66415ea5d5f876"
server_key_iv: "4006edaf0250f72db850a6eb027da3f1"
server_key_version: "kuduclusterkey@0"
I20260929 10:35:37.165288 28661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:35:37.182214 28661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:35:37.185864 28661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:35:37.186064 28661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:35:37.186775 28661 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:35:37.187747 28661 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:35:37.187808 28661 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:35:37.187862 28661 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:35:37.187891 28661 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:35:37.200750 28661 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.66:42351
I20260929 10:35:37.201349 28790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.66:42351 every 8 connection(s)
I20260929 10:35:37.201949 28661 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
I20260929 10:35:37.202668 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 28661
I20260929 10:35:37.203130 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.82.84:34645/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.67:33605
--local_ip_for_outbound_sockets=127.15.82.67
--tserver_master_addrs=127.15.82.126:33739
--webserver_port=38957
--webserver_interface=127.15.82.67
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab
--principal=kudu/127.15.82.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:44449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.15.82.84:34645/kms with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 10:35:37 dist-test-slave-vg89 krb5kdc[27463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678137, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.66@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:35:37.229139 28659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:37.206219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:59937 (local address 127.15.82.126:33739)
0929 10:35:37.206352 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0929 10:35:37.206356 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:35:37.208127 (+ 1771us) server_negotiation.cc:407] Connection header received
0929 10:35:37.209681 (+ 1554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:35:37.209688 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:35:37.209762 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:35:37.209827 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:35:37.211795 (+ 1968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:37.212772 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:37.214818 (+ 2046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:37.215266 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:37.215362 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:37.224320 (+ 8958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:35:37.224327 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:35:37.224331 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:35:37.224359 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:35:37.226696 (+ 2337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:35:37.227316 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:35:37.227320 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:35:37.227322 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:35:37.227407 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:35:37.227815 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:35:37.227819 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:35:37.227821 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:35:37.228059 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:35:37.228139 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0929 10:35:37.228802 (+ 663us) server_negotiation.cc:299] Negotiation successful
0929 10:35:37.228950 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 10:35:37.230753 28794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:37.206742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:33739 (local address 127.15.82.66:59937)
0929 10:35:37.207947 (+ 1205us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:37.208016 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0929 10:35:37.209393 (+ 1377us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:35:37.210307 (+ 914us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:35:37.210319 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:35:37.210742 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:35:37.211602 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:37.211625 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:37.213026 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:37.213031 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:37.214661 (+ 1630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:37.214675 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:37.215591 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:37.215599 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:37.215830 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:37.222116 (+ 6286us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:35:37.222137 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:35:37.224073 (+ 1936us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:35:37.226832 (+ 2759us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:35:37.226838 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:35:37.226855 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:35:37.227204 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:35:37.227511 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:35:37.227514 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:35:37.227516 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:35:37.227704 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:35:37.228203 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:35:37.228209 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:35:37.228530 (+ 321us) client_negotiation.cc:769] Sending connection context
0929 10:35:37.229321 (+ 791us) client_negotiation.cc:240] Negotiation successful
0929 10:35:37.229672 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":961,"thread_start_us":153,"threads_started":1}
I20260929 10:35:37.232034 28791 heartbeater.cc:347] Connected to a master server at 127.15.82.126:33739
I20260929 10:35:37.232288 28791 heartbeater.cc:464] Registering TS with master...
I20260929 10:35:37.232852 28791 heartbeater.cc:511] Master 127.15.82.126:33739 requested a full tablet report, sending...
W20260929 10:35:37.234476 28791 heartbeater.cc:644] Failed to heartbeat to 127.15.82.126:33739 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260929 10:35:37.234597 28791 heartbeater.cc:464] Registering TS with master...
I20260929 10:35:37.234759 28791 heartbeater.cc:511] Master 127.15.82.126:33739 requested a full tablet report, sending...
I20260929 10:35:37.235491 28791 heartbeater.cc:464] Registering TS with master...
I20260929 10:35:37.235652 28791 heartbeater.cc:511] Master 127.15.82.126:33739 requested a full tablet report, sending...
W20260929 10:35:37.236330 28791 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 10:35:37.341240 28793 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:35:37.341609 28793 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:35:37.341678 28793 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:35:37.347740 28793 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:35:37.347815 28793 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:35:37.347903 28793 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.67
I20260929 10:35:37.355756 28793 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:44449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.15.82.84:34645/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab
--principal=kudu/127.15.82.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.67:33605
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.82.67
--webserver_port=38957
--tserver_master_addrs=127.15.82.126:33739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28793
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.67
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:35:37.357172 28793 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:35:37.358566 28793 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:35:37.369645 28800 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:35:37.369685 28801 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:35:37.369685 28803 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:35:37.370466 28793 server_base.cc:1034] running on GCE node
I20260929 10:35:37.370883 28793 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:35:37.371526 28793 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:35:37.372705 28793 hybrid_clock.cc:648] HybridClock initialized: now 1790678137372669 us; error 55 us; skew 500 ppm
Sep 29 10:35:37 dist-test-slave-vg89 krb5kdc[27463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678137, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:35:37.376237 28793 init.cc:377] Logged in from keytab as kudu/127.15.82.67@KRBTEST.COM (short username kudu)
I20260929 10:35:37.377382 28793 webserver.cc:460] Webserver started at http://127.15.82.67:38957/ using document root <none> and password file <none>
I20260929 10:35:37.378154 28793 fs_manager.cc:359] Metadata directory not provided
I20260929 10:35:37.378298 28793 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 29 10:35:37 dist-test-slave-vg89 krb5kdc[27463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678137, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.67@KRBTEST.COM for HTTP/127.15.82.84@KRBTEST.COM
I20260929 10:35:37.407303 28793 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.003s sys 0.000s
I20260929 10:35:37.412246 28811 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:35:37.413873 28793 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260929 10:35:37.414033 28793 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-2/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-2/wal
uuid: "76b89a15d6834c66b8f4932be823bd78"
format_stamp: "Formatted at 2026-09-29 10:35:36 on dist-test-slave-vg89"
server_key: "48739dbe4524e9f58c7cf50433e090e6"
server_key_iv: "07cd96e0034d4bef871600a4dfa5ca3f"
server_key_version: "kuduclusterkey@0"
I20260929 10:35:37.414721 28793 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:35:37.442016 28793 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:35:37.445654 28793 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:35:37.445861 28793 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:35:37.446524 28793 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:35:37.447371 28793 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:35:37.447446 28793 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:35:37.447512 28793 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:35:37.447542 28793 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:35:37.458144 28793 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.67:33605
I20260929 10:35:37.458173 28924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.67:33605 every 8 connection(s)
I20260929 10:35:37.459460 28793 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
I20260929 10:35:37.464004 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 28793
Sep 29 10:35:37 dist-test-slave-vg89 krb5kdc[27463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678137, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.67@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:35:37.479012 28929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:37.464757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52128 (local address 127.15.82.126:33739)
0929 10:35:37.465004 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0929 10:35:37.465014 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 10:35:37.468680 (+ 3666us) server_negotiation.cc:407] Connection header received
0929 10:35:37.468920 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:35:37.468923 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:35:37.468989 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:35:37.469058 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:35:37.470340 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:37.471210 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:37.472424 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:37.472671 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:37.472777 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:37.474726 (+ 1949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:35:37.474732 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:35:37.474736 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:35:37.475051 (+ 315us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:35:37.477032 (+ 1981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:35:37.477599 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:35:37.477603 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:35:37.477605 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:35:37.477695 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:35:37.478015 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:35:37.478019 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:35:37.478021 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:35:37.478218 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:35:37.478362 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0929 10:35:37.478660 (+ 298us) server_negotiation.cc:299] Negotiation successful
0929 10:35:37.478784 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":72,"threads_started":1}
I20260929 10:35:37.481505 28659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:37.464054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.67:56031 (local address 127.15.82.126:33739)
0929 10:35:37.464565 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0929 10:35:37.464570 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:35:37.467038 (+ 2468us) server_negotiation.cc:407] Connection header received
0929 10:35:37.468332 (+ 1294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:35:37.468337 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:35:37.468395 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:35:37.468487 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:35:37.470699 (+ 2212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:37.471490 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:37.473627 (+ 2137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:37.473978 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:37.474052 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:37.476949 (+ 2897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:35:37.476955 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:35:37.476959 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:35:37.477005 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:35:37.479066 (+ 2061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:35:37.479559 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:35:37.479562 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:35:37.479564 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:35:37.479641 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:35:37.480318 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:35:37.480321 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:35:37.480323 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:35:37.480515 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:35:37.480613 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0929 10:35:37.481318 (+ 705us) server_negotiation.cc:299] Negotiation successful
0929 10:35:37.481405 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"spinlock_wait_cycles":1792}
I20260929 10:35:37.482965 28927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:37.464547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:33739 (local address 127.15.82.67:56031)
0929 10:35:37.466861 (+ 2314us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:37.466919 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0929 10:35:37.468064 (+ 1145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:35:37.469232 (+ 1168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:35:37.469243 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:35:37.469842 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:35:37.470525 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:37.470555 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:37.472089 (+ 1534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:37.472093 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:37.473476 (+ 1383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:37.473486 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:37.474230 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:37.474234 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:37.474493 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:37.475117 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:35:37.475138 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:35:37.476787 (+ 1649us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:35:37.479194 (+ 2407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:35:37.479199 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:35:37.479211 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:35:37.479446 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:35:37.479833 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:35:37.479837 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:35:37.479839 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:35:37.480192 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:35:37.480633 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:35:37.480641 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:35:37.481053 (+ 412us) client_negotiation.cc:769] Sending connection context
0929 10:35:37.481271 (+ 218us) client_negotiation.cc:240] Negotiation successful
0929 10:35:37.481852 (+ 581us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2085,"thread_start_us":134,"threads_started":1}
I20260929 10:35:37.484095 28925 heartbeater.cc:347] Connected to a master server at 127.15.82.126:33739
I20260929 10:35:37.484330 28925 heartbeater.cc:464] Registering TS with master...
I20260929 10:35:37.484889 28925 heartbeater.cc:511] Master 127.15.82.126:33739 requested a full tablet report, sending...
W20260929 10:35:37.486517 28925 heartbeater.cc:644] Failed to heartbeat to 127.15.82.126:33739 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260929 10:35:37.486639 28925 heartbeater.cc:464] Registering TS with master...
I20260929 10:35:37.486800 28925 heartbeater.cc:511] Master 127.15.82.126:33739 requested a full tablet report, sending...
I20260929 10:35:37.487574 28925 heartbeater.cc:464] Registering TS with master...
I20260929 10:35:37.487740 28925 heartbeater.cc:511] Master 127.15.82.126:33739 requested a full tablet report, sending...
W20260929 10:35:37.488515 28925 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 29 10:35:37 dist-test-slave-vg89 krb5kdc[27463](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790678137, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:35:37.974282 28657 heartbeater.cc:464] Registering TS with master...
I20260929 10:35:37.974637 28657 heartbeater.cc:511] Master 127.15.82.126:33739 requested a full tablet report, sending...
I20260929 10:35:37.975668 28459 ts_manager.cc:194] Registered new tserver with Master: 814e446a7d1e44589801e2242186d5d0 (127.15.82.65:42219)
I20260929 10:35:37.976991 28459 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.82.65@KRBTEST.COM'} at 127.15.82.65:46611
I20260929 10:35:38.236559 28791 heartbeater.cc:464] Registering TS with master...
I20260929 10:35:38.236860 28791 heartbeater.cc:511] Master 127.15.82.126:33739 requested a full tablet report, sending...
I20260929 10:35:38.237603 28459 ts_manager.cc:194] Registered new tserver with Master: 492cde4f6775496bbad5555ed4d806f4 (127.15.82.66:42351)
I20260929 10:35:38.238296 28459 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.82.66@KRBTEST.COM'} at 127.15.82.66:59937
Sep 29 10:35:38 dist-test-slave-vg89 krb5kdc[27463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678137, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.126@KRBTEST.COM for HTTP/127.15.82.84@KRBTEST.COM
I20260929 10:35:38.488729 28925 heartbeater.cc:464] Registering TS with master...
I20260929 10:35:38.489045 28925 heartbeater.cc:511] Master 127.15.82.126:33739 requested a full tablet report, sending...
I20260929 10:35:38.490129 28459 ts_manager.cc:194] Registered new tserver with Master: 76b89a15d6834c66b8f4932be823bd78 (127.15.82.67:33605)
I20260929 10:35:38.490921 28459 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.82.67@KRBTEST.COM'} at 127.15.82.67:56031
I20260929 10:35:38.501410 15689 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 10:35:38.513615 28947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:38.503093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52140 (local address 127.15.82.126:33739)
0929 10:35:38.503739 (+ 646us) server_negotiation.cc:206] Beginning negotiation
0929 10:35:38.503745 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 10:35:38.504118 (+ 373us) server_negotiation.cc:407] Connection header received
0929 10:35:38.504160 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:35:38.504163 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:35:38.504210 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:35:38.504285 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:35:38.505217 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:38.506080 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:38.507328 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:38.507676 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:38.507788 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:38.509546 (+ 1758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:35:38.509565 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:35:38.509571 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:35:38.509634 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:35:38.511415 (+ 1781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:35:38.511899 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:35:38.511902 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:35:38.511904 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:35:38.512000 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:35:38.512402 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:35:38.512405 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:35:38.512407 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:35:38.512620 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:35:38.512816 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0929 10:35:38.513309 (+ 493us) server_negotiation.cc:299] Negotiation successful
0929 10:35:38.513428 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":90,"threads_started":1}
Sep 29 10:35:38 dist-test-slave-vg89 krb5kdc[27463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678137, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.126@KRBTEST.COM for HTTP/127.15.82.84@KRBTEST.COM
I20260929 10:35:38.517995 28459 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52140:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 10:35:38.526487 28459 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:35:38.568614 28954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:38.560452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.65:42219 (local address 127.0.0.1:54974)
0929 10:35:38.562284 (+ 1832us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:38.562336 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0929 10:35:38.562498 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:35:38.563281 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:35:38.563287 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:35:38.563310 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:35:38.563440 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:38.563447 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:38.565020 (+ 1573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:38.565023 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:38.566489 (+ 1466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:38.566500 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:38.568045 (+ 1545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:38.568062 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:38.568180 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:38.568219 (+ 39us) client_negotiation.cc:769] Sending connection context
0929 10:35:38.568328 (+ 109us) client_negotiation.cc:240] Negotiation successful
0929 10:35:38.568447 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1663,"thread_start_us":106,"threads_started":1}
I20260929 10:35:38.569489 28956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:38.561262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54974 (local address 127.15.82.65:42219)
0929 10:35:38.562010 (+ 748us) server_negotiation.cc:206] Beginning negotiation
0929 10:35:38.562018 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 10:35:38.562653 (+ 635us) server_negotiation.cc:407] Connection header received
0929 10:35:38.562741 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:35:38.562747 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:35:38.562953 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:35:38.563105 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:35:38.563574 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:38.564889 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:38.566626 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:38.567722 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:38.567826 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:38.567983 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0929 10:35:38.569241 (+ 1258us) server_negotiation.cc:299] Negotiation successful
0929 10:35:38.569348 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":633,"thread_start_us":61,"threads_started":1}
I20260929 10:35:38.573845 28591 tablet_service.cc:1467] Processing CreateTablet for tablet a98278ebfa714a709e6e10cb607751ce (DEFAULT_TABLE table=test-table [id=4f979457b57d4be5b0fb8ba67bf49a6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:35:38.574913 28954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:38.562003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.67:33605 (local address 127.0.0.1:52874)
0929 10:35:38.569026 (+ 7023us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:38.569040 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 10:35:38.569119 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:35:38.570694 (+ 1575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:35:38.570697 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:35:38.570715 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:35:38.570797 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:38.570807 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:38.572157 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:38.572160 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:38.573205 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:38.573216 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:38.574510 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:38.574515 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:38.574600 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:38.574625 (+ 25us) client_negotiation.cc:769] Sending connection context
0929 10:35:38.574700 (+ 75us) client_negotiation.cc:240] Negotiation successful
0929 10:35:38.574786 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6967,"thread_start_us":111,"threads_started":1}
I20260929 10:35:38.575098 28591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a98278ebfa714a709e6e10cb607751ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:35:38.575664 28959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:38.562440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52874 (local address 127.15.82.67:33605)
0929 10:35:38.569985 (+ 7545us) server_negotiation.cc:206] Beginning negotiation
0929 10:35:38.569993 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 10:35:38.570018 (+ 25us) server_negotiation.cc:407] Connection header received
0929 10:35:38.570109 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:35:38.570114 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:35:38.570322 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:35:38.570539 (+ 217us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:35:38.570930 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:38.572021 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:38.573341 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:38.574104 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:38.574201 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:38.574337 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0929 10:35:38.575433 (+ 1096us) server_negotiation.cc:299] Negotiation successful
0929 10:35:38.575534 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7433,"thread_start_us":107,"threads_started":1}
I20260929 10:35:38.577509 28958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:38.560530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.66:42351 (local address 127.0.0.1:46698)
0929 10:35:38.566575 (+ 6045us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:38.566591 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 10:35:38.566687 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:35:38.573197 (+ 6510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:35:38.573200 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:35:38.573218 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:35:38.573312 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:38.573318 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:38.574904 (+ 1586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:38.574907 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:38.576014 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:38.576026 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:38.577129 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:38.577134 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:38.577229 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:38.577253 (+ 24us) client_negotiation.cc:769] Sending connection context
0929 10:35:38.577302 (+ 49us) client_negotiation.cc:240] Negotiation successful
0929 10:35:38.577366 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5958,"thread_start_us":109,"threads_started":1}
I20260929 10:35:38.578259 28957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:38.561282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46698 (local address 127.15.82.66:42351)
0929 10:35:38.572608 (+ 11326us) server_negotiation.cc:206] Beginning negotiation
0929 10:35:38.572615 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 10:35:38.572642 (+ 27us) server_negotiation.cc:407] Connection header received
0929 10:35:38.572724 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:35:38.572730 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:35:38.572938 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:35:38.573471 (+ 533us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:35:38.573636 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:38.574770 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:38.576171 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:38.576987 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:38.577819 (+ 832us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:38.577975 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0929 10:35:38.578025 (+ 50us) server_negotiation.cc:299] Negotiation successful
0929 10:35:38.578119 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11200,"thread_start_us":105,"threads_started":1}
I20260929 10:35:38.578759 28855 tablet_service.cc:1467] Processing CreateTablet for tablet a98278ebfa714a709e6e10cb607751ce (DEFAULT_TABLE table=test-table [id=4f979457b57d4be5b0fb8ba67bf49a6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:35:38.579794 28855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a98278ebfa714a709e6e10cb607751ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:35:38.580644 28714 tablet_service.cc:1467] Processing CreateTablet for tablet a98278ebfa714a709e6e10cb607751ce (DEFAULT_TABLE table=test-table [id=4f979457b57d4be5b0fb8ba67bf49a6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:35:38.581362 28714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a98278ebfa714a709e6e10cb607751ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:35:38.591722 28962 tablet_bootstrap.cc:492] T a98278ebfa714a709e6e10cb607751ce P 76b89a15d6834c66b8f4932be823bd78: Bootstrap starting.
I20260929 10:35:38.593825 28962 tablet_bootstrap.cc:654] T a98278ebfa714a709e6e10cb607751ce P 76b89a15d6834c66b8f4932be823bd78: Neither blocks nor log segments found. Creating new log.
I20260929 10:35:38.594779 28962 log.cc:824] T a98278ebfa714a709e6e10cb607751ce P 76b89a15d6834c66b8f4932be823bd78: Log is configured to *not* fsync() on all Append() calls
I20260929 10:35:38.595224 28961 tablet_bootstrap.cc:492] T a98278ebfa714a709e6e10cb607751ce P 814e446a7d1e44589801e2242186d5d0: Bootstrap starting.
I20260929 10:35:38.597393 28961 tablet_bootstrap.cc:654] T a98278ebfa714a709e6e10cb607751ce P 814e446a7d1e44589801e2242186d5d0: Neither blocks nor log segments found. Creating new log.
I20260929 10:35:38.597862 28960 tablet_bootstrap.cc:492] T a98278ebfa714a709e6e10cb607751ce P 492cde4f6775496bbad5555ed4d806f4: Bootstrap starting.
I20260929 10:35:38.598207 28961 log.cc:824] T a98278ebfa714a709e6e10cb607751ce P 814e446a7d1e44589801e2242186d5d0: Log is configured to *not* fsync() on all Append() calls
I20260929 10:35:38.600051 28960 tablet_bootstrap.cc:654] T a98278ebfa714a709e6e10cb607751ce P 492cde4f6775496bbad5555ed4d806f4: Neither blocks nor log segments found. Creating new log.
I20260929 10:35:38.600872 28960 log.cc:824] T a98278ebfa714a709e6e10cb607751ce P 492cde4f6775496bbad5555ed4d806f4: Log is configured to *not* fsync() on all Append() calls
I20260929 10:35:38.601452 28962 tablet_bootstrap.cc:492] T a98278ebfa714a709e6e10cb607751ce P 76b89a15d6834c66b8f4932be823bd78: No bootstrap required, opened a new log
I20260929 10:35:38.601655 28962 ts_tablet_manager.cc:1397] T a98278ebfa714a709e6e10cb607751ce P 76b89a15d6834c66b8f4932be823bd78: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.000s
I20260929 10:35:38.603569 28960 tablet_bootstrap.cc:492] T a98278ebfa714a709e6e10cb607751ce P 492cde4f6775496bbad5555ed4d806f4: No bootstrap required, opened a new log
I20260929 10:35:38.603762 28960 ts_tablet_manager.cc:1397] T a98278ebfa714a709e6e10cb607751ce P 492cde4f6775496bbad5555ed4d806f4: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260929 10:35:38.605705 28962 raft_consensus.cc:348] T a98278ebfa714a709e6e10cb607751ce P 76b89a15d6834c66b8f4932be823bd78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "814e446a7d1e44589801e2242186d5d0" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 42219 } } peers { permanent_uuid: "492cde4f6775496bbad5555ed4d806f4" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 42351 } } peers { permanent_uuid: "76b89a15d6834c66b8f4932be823bd78" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 33605 } }
I20260929 10:35:38.605974 28962 raft_consensus.cc:374] T a98278ebfa714a709e6e10cb607751ce P 76b89a15d6834c66b8f4932be823bd78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:35:38.606029 28962 raft_consensus.cc:729] T a98278ebfa714a709e6e10cb607751ce P 76b89a15d6834c66b8f4932be823bd78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76b89a15d6834c66b8f4932be823bd78, State: Initialized, Role: FOLLOWER
I20260929 10:35:38.606606 28962 consensus_queue.cc:260] T a98278ebfa714a709e6e10cb607751ce P 76b89a15d6834c66b8f4932be823bd78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "814e446a7d1e44589801e2242186d5d0" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 42219 } } peers { permanent_uuid: "492cde4f6775496bbad5555ed4d806f4" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 42351 } } peers { permanent_uuid: "76b89a15d6834c66b8f4932be823bd78" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 33605 } }
I20260929 10:35:38.607698 28962 ts_tablet_manager.cc:1428] T a98278ebfa714a709e6e10cb607751ce P 76b89a15d6834c66b8f4932be823bd78: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260929 10:35:38.608049 28925 heartbeater.cc:503] Master 127.15.82.126:33739 was elected leader, sending a full tablet report...
I20260929 10:35:38.607990 28960 raft_consensus.cc:348] T a98278ebfa714a709e6e10cb607751ce P 492cde4f6775496bbad5555ed4d806f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "814e446a7d1e44589801e2242186d5d0" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 42219 } } peers { permanent_uuid: "492cde4f6775496bbad5555ed4d806f4" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 42351 } } peers { permanent_uuid: "76b89a15d6834c66b8f4932be823bd78" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 33605 } }
I20260929 10:35:38.608371 28960 raft_consensus.cc:374] T a98278ebfa714a709e6e10cb607751ce P 492cde4f6775496bbad5555ed4d806f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:35:38.608476 28960 raft_consensus.cc:729] T a98278ebfa714a709e6e10cb607751ce P 492cde4f6775496bbad5555ed4d806f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 492cde4f6775496bbad5555ed4d806f4, State: Initialized, Role: FOLLOWER
I20260929 10:35:38.608989 28960 consensus_queue.cc:260] T a98278ebfa714a709e6e10cb607751ce P 492cde4f6775496bbad5555ed4d806f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "814e446a7d1e44589801e2242186d5d0" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 42219 } } peers { permanent_uuid: "492cde4f6775496bbad5555ed4d806f4" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 42351 } } peers { permanent_uuid: "76b89a15d6834c66b8f4932be823bd78" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 33605 } }
I20260929 10:35:38.610025 28960 ts_tablet_manager.cc:1428] T a98278ebfa714a709e6e10cb607751ce P 492cde4f6775496bbad5555ed4d806f4: Time spent starting tablet: real 0.006s user 0.002s sys 0.003s
I20260929 10:35:38.610446 28791 heartbeater.cc:503] Master 127.15.82.126:33739 was elected leader, sending a full tablet report...
I20260929 10:35:38.613906 28961 tablet_bootstrap.cc:492] T a98278ebfa714a709e6e10cb607751ce P 814e446a7d1e44589801e2242186d5d0: No bootstrap required, opened a new log
I20260929 10:35:38.614126 28961 ts_tablet_manager.cc:1397] T a98278ebfa714a709e6e10cb607751ce P 814e446a7d1e44589801e2242186d5d0: Time spent bootstrapping tablet: real 0.019s user 0.004s sys 0.000s
I20260929 10:35:38.619184 28961 raft_consensus.cc:348] T a98278ebfa714a709e6e10cb607751ce P 814e446a7d1e44589801e2242186d5d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "814e446a7d1e44589801e2242186d5d0" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 42219 } } peers { permanent_uuid: "492cde4f6775496bbad5555ed4d806f4" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 42351 } } peers { permanent_uuid: "76b89a15d6834c66b8f4932be823bd78" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 33605 } }
I20260929 10:35:38.619554 28961 raft_consensus.cc:374] T a98278ebfa714a709e6e10cb607751ce P 814e446a7d1e44589801e2242186d5d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:35:38.619625 28961 raft_consensus.cc:729] T a98278ebfa714a709e6e10cb607751ce P 814e446a7d1e44589801e2242186d5d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 814e446a7d1e44589801e2242186d5d0, State: Initialized, Role: FOLLOWER
I20260929 10:35:38.620162 28961 consensus_queue.cc:260] T a98278ebfa714a709e6e10cb607751ce P 814e446a7d1e44589801e2242186d5d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "814e446a7d1e44589801e2242186d5d0" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 42219 } } peers { permanent_uuid: "492cde4f6775496bbad5555ed4d806f4" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 42351 } } peers { permanent_uuid: "76b89a15d6834c66b8f4932be823bd78" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 33605 } }
I20260929 10:35:38.621203 28961 ts_tablet_manager.cc:1428] T a98278ebfa714a709e6e10cb607751ce P 814e446a7d1e44589801e2242186d5d0: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260929 10:35:38.621534 28657 heartbeater.cc:503] Master 127.15.82.126:33739 was elected leader, sending a full tablet report...
W20260929 10:35:38.691076 28658 tablet.cc:2369] T a98278ebfa714a709e6e10cb607751ce P 814e446a7d1e44589801e2242186d5d0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 10:35:38.712603 28926 tablet.cc:2369] T a98278ebfa714a709e6e10cb607751ce P 76b89a15d6834c66b8f4932be823bd78: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 10:35:38.714203 28792 tablet.cc:2369] T a98278ebfa714a709e6e10cb607751ce P 492cde4f6775496bbad5555ed4d806f4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 10:35:38.723466 28967 raft_consensus.cc:482] T a98278ebfa714a709e6e10cb607751ce P 492cde4f6775496bbad5555ed4d806f4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:35:38.723697 28967 raft_consensus.cc:504] T a98278ebfa714a709e6e10cb607751ce P 492cde4f6775496bbad5555ed4d806f4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "814e446a7d1e44589801e2242186d5d0" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 42219 } } peers { permanent_uuid: "492cde4f6775496bbad5555ed4d806f4" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 42351 } } peers { permanent_uuid: "76b89a15d6834c66b8f4932be823bd78" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 33605 } }
I20260929 10:35:38.725178 28967 leader_election.cc:288] T a98278ebfa714a709e6e10cb607751ce P 492cde4f6775496bbad5555ed4d806f4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 814e446a7d1e44589801e2242186d5d0 (127.15.82.65:42219), 76b89a15d6834c66b8f4932be823bd78 (127.15.82.67:33605)
I20260929 10:35:38.737425 28956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:38.726637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:54101 (local address 127.15.82.65:42219)
0929 10:35:38.726908 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0929 10:35:38.726913 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:35:38.728528 (+ 1615us) server_negotiation.cc:407] Connection header received
0929 10:35:38.728752 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:35:38.728756 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:35:38.728805 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:35:38.728854 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:35:38.729670 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:38.730497 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:38.735019 (+ 4522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:38.735654 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:38.735749 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:38.735802 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0929 10:35:38.737196 (+ 1394us) server_negotiation.cc:299] Negotiation successful
0929 10:35:38.737281 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180}
I20260929 10:35:38.737887 28969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:38.725593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.65:42219 (local address 127.15.82.66:54101)
0929 10:35:38.728425 (+ 2832us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:38.728441 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 10:35:38.728625 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:35:38.729000 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:35:38.729003 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:35:38.729020 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:35:38.729167 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:38.729175 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:38.730650 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:38.730654 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:38.732025 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:38.732035 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:38.735893 (+ 3858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:38.735897 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:38.735995 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:38.736017 (+ 22us) client_negotiation.cc:769] Sending connection context
0929 10:35:38.737646 (+ 1629us) client_negotiation.cc:240] Negotiation successful
0929 10:35:38.737726 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2750,"thread_start_us":133,"threads_started":1}
I20260929 10:35:38.738564 28611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a98278ebfa714a709e6e10cb607751ce" candidate_uuid: "492cde4f6775496bbad5555ed4d806f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "814e446a7d1e44589801e2242186d5d0" is_pre_election: true
I20260929 10:35:38.738937 28611 raft_consensus.cc:2454] T a98278ebfa714a709e6e10cb607751ce P 814e446a7d1e44589801e2242186d5d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 492cde4f6775496bbad5555ed4d806f4 in term 0.
I20260929 10:35:38.739470 28678 leader_election.cc:302] T a98278ebfa714a709e6e10cb607751ce P 492cde4f6775496bbad5555ed4d806f4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 492cde4f6775496bbad5555ed4d806f4, 814e446a7d1e44589801e2242186d5d0; no voters:
I20260929 10:35:38.739780 28967 raft_consensus.cc:2781] T a98278ebfa714a709e6e10cb607751ce P 492cde4f6775496bbad5555ed4d806f4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:35:38.739881 28967 raft_consensus.cc:482] T a98278ebfa714a709e6e10cb607751ce P 492cde4f6775496bbad5555ed4d806f4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:35:38.739924 28967 raft_consensus.cc:3037] T a98278ebfa714a709e6e10cb607751ce P 492cde4f6775496bbad5555ed4d806f4 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:35:38.740087 28959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:38.726770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:34691 (local address 127.15.82.67:33605)
0929 10:35:38.727002 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0929 10:35:38.727007 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:35:38.729489 (+ 2482us) server_negotiation.cc:407] Connection header received
0929 10:35:38.732185 (+ 2696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:35:38.732189 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:35:38.732240 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:35:38.732287 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:35:38.732659 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:38.733447 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:38.736129 (+ 2682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:38.736743 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:38.736832 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:38.736877 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0929 10:35:38.739877 (+ 3000us) server_negotiation.cc:299] Negotiation successful
0929 10:35:38.739966 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162}
I20260929 10:35:38.740516 28970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:38.726033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.67:33605 (local address 127.15.82.66:34691)
0929 10:35:38.729411 (+ 3378us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:38.729424 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 10:35:38.729560 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:35:38.732410 (+ 2850us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:35:38.732412 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:35:38.732428 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:35:38.732529 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:38.732534 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:38.733597 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:38.733600 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:38.734878 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:38.734887 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:38.736965 (+ 2078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:38.736969 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:38.737067 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:38.737086 (+ 19us) client_negotiation.cc:769] Sending connection context
0929 10:35:38.740302 (+ 3216us) client_negotiation.cc:240] Negotiation successful
0929 10:35:38.740379 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3317,"thread_start_us":92,"threads_started":1}
I20260929 10:35:38.741101 28870 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a98278ebfa714a709e6e10cb607751ce" candidate_uuid: "492cde4f6775496bbad5555ed4d806f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76b89a15d6834c66b8f4932be823bd78" is_pre_election: true
I20260929 10:35:38.741354 28967 raft_consensus.cc:504] T a98278ebfa714a709e6e10cb607751ce P 492cde4f6775496bbad5555ed4d806f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "814e446a7d1e44589801e2242186d5d0" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 42219 } } peers { permanent_uuid: "492cde4f6775496bbad5555ed4d806f4" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 42351 } } peers { permanent_uuid: "76b89a15d6834c66b8f4932be823bd78" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 33605 } }
I20260929 10:35:38.741428 28870 raft_consensus.cc:2454] T a98278ebfa714a709e6e10cb607751ce P 76b89a15d6834c66b8f4932be823bd78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 492cde4f6775496bbad5555ed4d806f4 in term 0.
I20260929 10:35:38.741724 28967 leader_election.cc:288] T a98278ebfa714a709e6e10cb607751ce P 492cde4f6775496bbad5555ed4d806f4 [CANDIDATE]: Term 1 election: Requested vote from peers 814e446a7d1e44589801e2242186d5d0 (127.15.82.65:42219), 76b89a15d6834c66b8f4932be823bd78 (127.15.82.67:33605)
I20260929 10:35:38.742523 28611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a98278ebfa714a709e6e10cb607751ce" candidate_uuid: "492cde4f6775496bbad5555ed4d806f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "814e446a7d1e44589801e2242186d5d0"
I20260929 10:35:38.742652 28611 raft_consensus.cc:3037] T a98278ebfa714a709e6e10cb607751ce P 814e446a7d1e44589801e2242186d5d0 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:35:38.743927 28611 raft_consensus.cc:2454] T a98278ebfa714a709e6e10cb607751ce P 814e446a7d1e44589801e2242186d5d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 492cde4f6775496bbad5555ed4d806f4 in term 1.
I20260929 10:35:38.742362 28870 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a98278ebfa714a709e6e10cb607751ce" candidate_uuid: "492cde4f6775496bbad5555ed4d806f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76b89a15d6834c66b8f4932be823bd78"
I20260929 10:35:38.744196 28678 leader_election.cc:302] T a98278ebfa714a709e6e10cb607751ce P 492cde4f6775496bbad5555ed4d806f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 492cde4f6775496bbad5555ed4d806f4, 814e446a7d1e44589801e2242186d5d0; no voters:
I20260929 10:35:38.744275 28870 raft_consensus.cc:3037] T a98278ebfa714a709e6e10cb607751ce P 76b89a15d6834c66b8f4932be823bd78 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:35:38.744442 28967 raft_consensus.cc:2781] T a98278ebfa714a709e6e10cb607751ce P 492cde4f6775496bbad5555ed4d806f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:35:38.744659 28967 raft_consensus.cc:686] T a98278ebfa714a709e6e10cb607751ce P 492cde4f6775496bbad5555ed4d806f4 [term 1 LEADER]: Becoming Leader. State: Replica: 492cde4f6775496bbad5555ed4d806f4, State: Running, Role: LEADER
I20260929 10:35:38.744990 28967 consensus_queue.cc:237] T a98278ebfa714a709e6e10cb607751ce P 492cde4f6775496bbad5555ed4d806f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "814e446a7d1e44589801e2242186d5d0" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 42219 } } peers { permanent_uuid: "492cde4f6775496bbad5555ed4d806f4" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 42351 } } peers { permanent_uuid: "76b89a15d6834c66b8f4932be823bd78" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 33605 } }
I20260929 10:35:38.745513 28870 raft_consensus.cc:2454] T a98278ebfa714a709e6e10cb607751ce P 76b89a15d6834c66b8f4932be823bd78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 492cde4f6775496bbad5555ed4d806f4 in term 1.
I20260929 10:35:38.748325 28459 catalog_manager.cc:5495] T a98278ebfa714a709e6e10cb607751ce P 492cde4f6775496bbad5555ed4d806f4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 492cde4f6775496bbad5555ed4d806f4 (127.15.82.66). New cstate: current_term: 1 leader_uuid: "492cde4f6775496bbad5555ed4d806f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "814e446a7d1e44589801e2242186d5d0" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 42219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "492cde4f6775496bbad5555ed4d806f4" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 42351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "76b89a15d6834c66b8f4932be823bd78" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 33605 } health_report { overall_health: UNKNOWN } } }
I20260929 10:35:38.801240 28957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:38.797094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46708 (local address 127.15.82.66:42351)
0929 10:35:38.797241 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0929 10:35:38.797246 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:35:38.797268 (+ 22us) server_negotiation.cc:407] Connection header received
0929 10:35:38.797361 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:35:38.797364 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:35:38.797431 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:35:38.797559 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:35:38.797766 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:38.798587 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:38.799445 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:38.799805 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:38.799920 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:38.800236 (+ 316us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:35:38.800834 (+ 598us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:35:38.800961 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0929 10:35:38.800996 (+ 35us) server_negotiation.cc:299] Negotiation successful
0929 10:35:38.801081 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260929 10:35:38.808135 28611 raft_consensus.cc:1261] T a98278ebfa714a709e6e10cb607751ce P 814e446a7d1e44589801e2242186d5d0 [term 1 FOLLOWER]: Refusing update from remote peer 492cde4f6775496bbad5555ed4d806f4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:35:38.808144 28870 raft_consensus.cc:1261] T a98278ebfa714a709e6e10cb607751ce P 76b89a15d6834c66b8f4932be823bd78 [term 1 FOLLOWER]: Refusing update from remote peer 492cde4f6775496bbad5555ed4d806f4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:35:38.808920 28967 consensus_queue.cc:1035] T a98278ebfa714a709e6e10cb607751ce P 492cde4f6775496bbad5555ed4d806f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76b89a15d6834c66b8f4932be823bd78" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 33605 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:35:38.809135 28971 consensus_queue.cc:1035] T a98278ebfa714a709e6e10cb607751ce P 492cde4f6775496bbad5555ed4d806f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "814e446a7d1e44589801e2242186d5d0" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 42219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:35:38.827873 28983 mvcc.cc:205] Tried to move back new op lower bound from 7334617656550047744 to 7334617656303153152. Current Snapshot: MvccSnapshot[applied={T|T < 7334617656550047744}]
I20260929 10:35:38.828171 28458 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52140:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:35:38.828338 28458 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52140:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:35:38.830802 28458 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a751487001594e6293a13894cf876e50: Sending DeleteTablet for 3 replicas of tablet a98278ebfa714a709e6e10cb607751ce
I20260929 10:35:38.831521 28591 tablet_service.cc:1514] Processing DeleteTablet for tablet a98278ebfa714a709e6e10cb607751ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:35:38 UTC) from {username='kudu', principal='kudu/127.15.82.126@KRBTEST.COM'} at 127.0.0.1:54974
I20260929 10:35:38.831663 28714 tablet_service.cc:1514] Processing DeleteTablet for tablet a98278ebfa714a709e6e10cb607751ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:35:38 UTC) from {username='kudu', principal='kudu/127.15.82.126@KRBTEST.COM'} at 127.0.0.1:46698
I20260929 10:35:38.832257 28991 tablet_replica.cc:330] T a98278ebfa714a709e6e10cb607751ce P 492cde4f6775496bbad5555ed4d806f4: stopping tablet replica
I20260929 10:35:38.832254 28855 tablet_service.cc:1514] Processing DeleteTablet for tablet a98278ebfa714a709e6e10cb607751ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:35:38 UTC) from {username='kudu', principal='kudu/127.15.82.126@KRBTEST.COM'} at 127.0.0.1:52874
I20260929 10:35:38.832624 28991 raft_consensus.cc:2229] T a98278ebfa714a709e6e10cb607751ce P 492cde4f6775496bbad5555ed4d806f4 [term 1 LEADER]: Raft consensus shutting down.
I20260929 10:35:38.832651 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 28497
I20260929 10:35:38.833835 28991 raft_consensus.cc:2258] T a98278ebfa714a709e6e10cb607751ce P 492cde4f6775496bbad5555ed4d806f4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:35:38.835402 28991 ts_tablet_manager.cc:1905] T a98278ebfa714a709e6e10cb607751ce P 492cde4f6775496bbad5555ed4d806f4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:35:38.838244 28991 ts_tablet_manager.cc:1918] T a98278ebfa714a709e6e10cb607751ce P 492cde4f6775496bbad5555ed4d806f4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:35:38.838392 28991 log.cc:1196] T a98278ebfa714a709e6e10cb607751ce P 492cde4f6775496bbad5555ed4d806f4: Deleting WAL directory at /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-1/wal/wals/a98278ebfa714a709e6e10cb607751ce
I20260929 10:35:38.838869 28991 ts_tablet_manager.cc:1939] T a98278ebfa714a709e6e10cb607751ce P 492cde4f6775496bbad5555ed4d806f4: Deleting consensus metadata
I20260929 10:35:38.840065 28446 catalog_manager.cc:4841] TS 492cde4f6775496bbad5555ed4d806f4 (127.15.82.66:42351): tablet a98278ebfa714a709e6e10cb607751ce (table test-table [id=4f979457b57d4be5b0fb8ba67bf49a6f]) successfully deleted
I20260929 10:35:38.842604 28992 tablet_replica.cc:330] T a98278ebfa714a709e6e10cb607751ce P 76b89a15d6834c66b8f4932be823bd78: stopping tablet replica
I20260929 10:35:38.842994 28992 raft_consensus.cc:2229] T a98278ebfa714a709e6e10cb607751ce P 76b89a15d6834c66b8f4932be823bd78 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260929 10:35:38.843075 28443 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.82.65:42219 (error 108)
I20260929 10:35:38.843402 28992 raft_consensus.cc:2258] T a98278ebfa714a709e6e10cb607751ce P 76b89a15d6834c66b8f4932be823bd78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:35:38.844156 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 28661
I20260929 10:35:38.845486 28992 ts_tablet_manager.cc:1905] T a98278ebfa714a709e6e10cb607751ce P 76b89a15d6834c66b8f4932be823bd78: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:35:38.848080 28992 ts_tablet_manager.cc:1918] T a98278ebfa714a709e6e10cb607751ce P 76b89a15d6834c66b8f4932be823bd78: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:35:38.848170 28992 log.cc:1196] T a98278ebfa714a709e6e10cb607751ce P 76b89a15d6834c66b8f4932be823bd78: Deleting WAL directory at /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790677981592896-15689-0/minicluster-data/ts-2/wal/wals/a98278ebfa714a709e6e10cb607751ce
I20260929 10:35:38.848489 28992 ts_tablet_manager.cc:1939] T a98278ebfa714a709e6e10cb607751ce P 76b89a15d6834c66b8f4932be823bd78: Deleting consensus metadata
I20260929 10:35:38.849053 28958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:38.848760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.65:42219 (local address 127.0.0.1:54986)
0929 10:35:38.848885 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:38.848970 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.65:42219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260929 10:35:38.849222 28443 catalog_manager.cc:4584] TS 814e446a7d1e44589801e2242186d5d0 (127.15.82.65:42219): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a98278ebfa714a709e6e10cb607751ce: Network error: Client connection negotiation failed: client connection to 127.15.82.65:42219: connect: Connection refused (error 111)
I20260929 10:35:38.850251 28445 catalog_manager.cc:4841] TS 76b89a15d6834c66b8f4932be823bd78 (127.15.82.67:33605): tablet a98278ebfa714a709e6e10cb607751ce (table test-table [id=4f979457b57d4be5b0fb8ba67bf49a6f]) successfully deleted
I20260929 10:35:38.851786 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 28793
I20260929 10:35:38.858047 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 28426
I20260929 10:35:38.865181 15689 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260929 10:35:39.200263 15689 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260929 10:35:39.200362 15689 mini_ranger.cc:66] Stopping Ranger...
I20260929 10:35:39.545553 15689 mini_ranger.cc:68] Stopped Ranger
2026-09-29 10:35:39.545 UTC [27493] LOG: received smart shutdown request
2026-09-29 10:35:39.549 UTC [27493] LOG: background worker "logical replication launcher" (PID 27502) exited with exit code 1
2026-09-29 10:35:39.549 UTC [27497] LOG: shutting down
2026-09-29 10:35:39.732 UTC [27493] LOG: database system is shut down
2026-09-29T10:35:39Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (49043 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790677981592896-15689-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:35:39 dist-test-slave-vg89 krb5kdc[29002](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:35:39 dist-test-slave-vg89 krb5kdc[29002](info): set up 2 sockets
Sep 29 10:35:39 dist-test-slave-vg89 krb5kdc[29002](info): commencing operation
krb5kdc: starting...
W20260929 10:35:40.910908 15689 mini_kdc.cc:121] Time spent starting KDC: real 1.033s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Sep 29 10:35:40 dist-test-slave-vg89 krb5kdc[29002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678140, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:35:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:35:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.82.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab.
Entry for principal kudu/127.15.82.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab.
I20260929 10:35:41.037949 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790677981592896-15689-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790677981592896-15689-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790677981592896-15689-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:35179
--webserver_interface=127.15.82.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab
--principal=kudu/127.15.82.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:43761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.82.126:35179,127.15.82.125:44731,127.15.82.124:34231
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:35:41.147507 29015 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:35:41.147819 29015 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:35:41.147904 29015 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:35:41.151863 29015 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:35:41.151978 29015 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:35:41.152020 29015 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:35:41.152077 29015 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:35:41.152115 29015 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:35:41.157305 29015 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:43761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790677981592896-15689-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790677981592896-15689-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.82.126:35179,127.15.82.125:44731,127.15.82.124:34231
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab
--principal=kudu/127.15.82.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:35179
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.82.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29015
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790677981592896-15689-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:35:41.158429 29015 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:35:41.159489 29015 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:35:41.167788 29023 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:35:41.167784 29021 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:35:41.167827 29020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:35:41.168064 29015 server_base.cc:1034] running on GCE node
I20260929 10:35:41.168744 29015 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:35:41.169919 29015 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:35:41.171118 29015 hybrid_clock.cc:648] HybridClock initialized: now 1790678141171103 us; error 27 us; skew 500 ppm
Sep 29 10:35:41 dist-test-slave-vg89 krb5kdc[29002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678141, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:35:41.174096 29015 init.cc:377] Logged in from keytab as kudu/127.15.82.126@KRBTEST.COM (short username kudu)
I20260929 10:35:41.175092 29015 webserver.cc:460] Webserver started at http://127.15.82.126:35009/ using document root <none> and password file <none>
I20260929 10:35:41.175702 29015 fs_manager.cc:359] Metadata directory not provided
I20260929 10:35:41.175797 29015 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:35:41.176041 29015 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:35:41.178210 29015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790677981592896-15689-0/minicluster-data/master-0/data/instance:
uuid: "6f2e41826e584a2e8864dd419989f112"
format_stamp: "Formatted at 2026-09-29 10:35:41 on dist-test-slave-vg89"
server_key: "52041a91021940af36b0b611e2f7bc29"
server_key_iv: "255eab34c187e3c685a62987b89ead21"
server_key_version: "encryptionkey@0"
I20260929 10:35:41.178867 29015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790677981592896-15689-0/minicluster-data/master-0/wal/instance:
uuid: "6f2e41826e584a2e8864dd419989f112"
format_stamp: "Formatted at 2026-09-29 10:35:41 on dist-test-slave-vg89"
server_key: "52041a91021940af36b0b611e2f7bc29"
server_key_iv: "255eab34c187e3c685a62987b89ead21"
server_key_version: "encryptionkey@0"
I20260929 10:35:41.182734 29015 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 10:35:41.185307 29030 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:35:41.186661 29015 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 10:35:41.186812 29015 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790677981592896-15689-0/minicluster-data/master-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790677981592896-15689-0/minicluster-data/master-0/wal
uuid: "6f2e41826e584a2e8864dd419989f112"
format_stamp: "Formatted at 2026-09-29 10:35:41 on dist-test-slave-vg89"
server_key: "52041a91021940af36b0b611e2f7bc29"
server_key_iv: "255eab34c187e3c685a62987b89ead21"
server_key_version: "encryptionkey@0"
I20260929 10:35:41.186942 29015 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790677981592896-15689-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790677981592896-15689-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790677981592896-15689-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:35:41.197711 29015 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:35:41.200655 29015 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:35:41.200874 29015 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:35:41.208761 29015 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.126:35179
I20260929 10:35:41.208792 29082 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.126:35179 every 8 connection(s)
I20260929 10:35:41.209743 29015 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
I20260929 10:35:41.212373 29083 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:35:41.214180 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 29015
I20260929 10:35:41.214303 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790677981592896-15689-0/minicluster-data/master-0/wal/instance
I20260929 10:35:41.214617 15689 external_mini_cluster.cc:1428] Setting key 782e30bb28336a851c9a9c3bc8dd9603
I20260929 10:35:41.217430 29083 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.82.126" port: 35179 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.15.82.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.125@KRBTEST.COM" created.
Sep 29 10:35:41 dist-test-slave-vg89 krb5kdc[29002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678141, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.126@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:35:41.234264 29086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:41.219870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:35179 (local address 127.0.0.1:50538)
0929 10:35:41.220451 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:41.220511 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0929 10:35:41.221688 (+ 1177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:35:41.222452 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:35:41.222461 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:35:41.222944 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:35:41.223335 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:41.223354 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:41.224561 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:41.224564 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:41.225388 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:41.225397 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:41.226166 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:41.226169 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:41.226267 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:41.227177 (+ 910us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:35:41.227194 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:35:41.228850 (+ 1656us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:35:41.231160 (+ 2310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:35:41.231171 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:35:41.231183 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:35:41.231514 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:35:41.231873 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:35:41.231879 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:35:41.231881 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:35:41.232056 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:35:41.232732 (+ 676us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:35:41.232740 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:35:41.232919 (+ 179us) client_negotiation.cc:769] Sending connection context
0929 10:35:41.233140 (+ 221us) client_negotiation.cc:240] Negotiation successful
0929 10:35:41.233508 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"mutex_wait_us":56,"thread_start_us":138,"threads_started":1}
I20260929 10:35:41.234266 29087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:41.219869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50538 (local address 127.15.82.126:35179)
0929 10:35:41.220451 (+ 582us) server_negotiation.cc:206] Beginning negotiation
0929 10:35:41.220471 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0929 10:35:41.220642 (+ 171us) server_negotiation.cc:407] Connection header received
0929 10:35:41.222016 (+ 1374us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:35:41.222021 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:35:41.222202 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:35:41.222366 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:35:41.223502 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:41.224438 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:41.225516 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:41.225990 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:41.226115 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:41.229032 (+ 2917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:35:41.229037 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:35:41.229048 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:35:41.229071 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:35:41.231032 (+ 1961us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:35:41.231645 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:35:41.231650 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:35:41.231651 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:35:41.231768 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:35:41.232188 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:35:41.232191 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:35:41.232192 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:35:41.232607 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:35:41.232719 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0929 10:35:41.233196 (+ 477us) server_negotiation.cc:299] Negotiation successful
0929 10:35:41.233428 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":443,"spinlock_wait_cycles":28032,"thread_start_us":160,"threads_started":1}
I20260929 10:35:41.236812 29083 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.82.125" port: 44731 } has no permanent_uuid. Determining permanent_uuid...
I20260929 10:35:41.237565 29086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:41.237328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.125:44731 (local address 127.0.0.1:47272)
0929 10:35:41.237440 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:41.237506 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.125:44731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260929 10:35:41.237748 29033 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.82.125:44731: connect: Connection refused (error 111)
I20260929 10:35:41.238708 29086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:41.238470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.125:44731 (local address 127.0.0.1:47286)
0929 10:35:41.238570 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:41.238649 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.125:44731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260929 10:35:41.238826 29083 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.82.125:44731: Network error: Client connection negotiation failed: client connection to 127.15.82.125:44731: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.125.keytab.
Entry for principal kudu/127.15.82.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.125.keytab.
I20260929 10:35:41.245512 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790677981592896-15689-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790677981592896-15689-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790677981592896-15689-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790677981592896-15689-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.125:44731
--webserver_interface=127.15.82.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.125.keytab
--principal=kudu/127.15.82.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:43761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.82.126:35179,127.15.82.125:44731,127.15.82.124:34231
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 10:35:41.288113 29083 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.82.125" port: 44731 } attempt: 1
I20260929 10:35:41.289050 29086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:41.288744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.125:44731 (local address 127.0.0.1:47298)
0929 10:35:41.288903 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:41.288972 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.125:44731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260929 10:35:41.289785 29086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:41.289568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.125:44731 (local address 127.0.0.1:47312)
0929 10:35:41.289677 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:41.289727 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.125:44731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260929 10:35:41.289911 29083 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.82.125:44731: Network error: Client connection negotiation failed: client connection to 127.15.82.125:44731: connect: Connection refused (error 111)
W20260929 10:35:41.355840 29090 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:35:41.356153 29090 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:35:41.356220 29090 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260929 10:35:41.358068 29083 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.82.125" port: 44731 } attempt: 2
I20260929 10:35:41.358920 29086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:41.358615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.125:44731 (local address 127.0.0.1:47326)
0929 10:35:41.358778 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:41.358845 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.125:44731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260929 10:35:41.359628 29086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:41.359412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.125:44731 (local address 127.0.0.1:47336)
0929 10:35:41.359520 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:41.359575 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.125:44731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260929 10:35:41.359751 29083 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.82.125:44731: Network error: Client connection negotiation failed: client connection to 127.15.82.125:44731: connect: Connection refused (error 111)
W20260929 10:35:41.360381 29090 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:35:41.360451 29090 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:35:41.360510 29090 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:35:41.360546 29090 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:35:41.360584 29090 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:35:41.365847 29090 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:43761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790677981592896-15689-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790677981592896-15689-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.15.82.126:35179,127.15.82.125:44731,127.15.82.124:34231
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.125.keytab
--principal=kudu/127.15.82.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.125:44731
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790677981592896-15689-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.15.82.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29090
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790677981592896-15689-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:35:41.366925 29090 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:35:41.367906 29090 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:35:41.376060 29096 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:35:41.376130 29095 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:35:41.376206 29098 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:35:41.376245 29090 server_base.cc:1034] running on GCE node
I20260929 10:35:41.376945 29090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:35:41.377995 29090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:35:41.379179 29090 hybrid_clock.cc:648] HybridClock initialized: now 1790678141379165 us; error 25 us; skew 500 ppm
Sep 29 10:35:41 dist-test-slave-vg89 krb5kdc[29002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678141, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:35:41.381836 29090 init.cc:377] Logged in from keytab as kudu/127.15.82.125@KRBTEST.COM (short username kudu)
I20260929 10:35:41.382794 29090 webserver.cc:460] Webserver started at http://127.15.82.125:41415/ using document root <none> and password file <none>
I20260929 10:35:41.383343 29090 fs_manager.cc:359] Metadata directory not provided
I20260929 10:35:41.383428 29090 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:35:41.383648 29090 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:35:41.385634 29090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790677981592896-15689-0/minicluster-data/master-1/data/instance:
uuid: "b222a401dd004f67b25de657fcd29257"
format_stamp: "Formatted at 2026-09-29 10:35:41 on dist-test-slave-vg89"
server_key: "68bcdcad002a4421bbd2155aa9e3a7b8"
server_key_iv: "6ca92a64be2048165da4586ea2608af9"
server_key_version: "encryptionkey@0"
I20260929 10:35:41.386185 29090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790677981592896-15689-0/minicluster-data/master-1/wal/instance:
uuid: "b222a401dd004f67b25de657fcd29257"
format_stamp: "Formatted at 2026-09-29 10:35:41 on dist-test-slave-vg89"
server_key: "68bcdcad002a4421bbd2155aa9e3a7b8"
server_key_iv: "6ca92a64be2048165da4586ea2608af9"
server_key_version: "encryptionkey@0"
I20260929 10:35:41.390046 29090 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260929 10:35:41.392674 29105 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:35:41.393904 29090 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 10:35:41.394052 29090 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790677981592896-15689-0/minicluster-data/master-1/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790677981592896-15689-0/minicluster-data/master-1/wal
uuid: "b222a401dd004f67b25de657fcd29257"
format_stamp: "Formatted at 2026-09-29 10:35:41 on dist-test-slave-vg89"
server_key: "68bcdcad002a4421bbd2155aa9e3a7b8"
server_key_iv: "6ca92a64be2048165da4586ea2608af9"
server_key_version: "encryptionkey@0"
I20260929 10:35:41.394179 29090 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790677981592896-15689-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790677981592896-15689-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790677981592896-15689-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:35:41.411532 29090 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:35:41.414541 29090 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:35:41.414748 29090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:35:41.421856 29090 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.125:44731
I20260929 10:35:41.421869 29157 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.125:44731 every 8 connection(s)
I20260929 10:35:41.422904 29090 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790677981592896-15689-0/minicluster-data/master-1/data/info.pb
I20260929 10:35:41.425626 29158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:35:41.429251 29158 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.82.126" port: 35179 } has no permanent_uuid. Determining permanent_uuid...
I20260929 10:35:41.432529 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 29090
I20260929 10:35:41.432637 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790677981592896-15689-0/minicluster-data/master-1/wal/instance
I20260929 10:35:41.432930 15689 external_mini_cluster.cc:1428] Setting key 4296f6872a006e0b91f83f7083c98d92
Sep 29 10:35:41 dist-test-slave-vg89 krb5kdc[29002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678141, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.125@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.82.124@KRBTEST.COM; defaulting to no policy
I20260929 10:35:41.442545 29087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:41.431136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50554 (local address 127.15.82.126:35179)
0929 10:35:41.431273 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0929 10:35:41.431277 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:35:41.432064 (+ 787us) server_negotiation.cc:407] Connection header received
0929 10:35:41.433034 (+ 970us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:35:41.433046 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:35:41.433110 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:35:41.433200 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:35:41.434318 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:41.434902 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:41.436057 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:41.436290 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:41.436398 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:41.438865 (+ 2467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:35:41.438868 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:35:41.438870 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:35:41.438896 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:35:41.440348 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:35:41.440911 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:35:41.440916 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:35:41.440918 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:35:41.441001 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:35:41.441341 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:35:41.441344 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:35:41.441346 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:35:41.441581 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:35:41.441683 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0929 10:35:41.442206 (+ 523us) server_negotiation.cc:299] Negotiation successful
0929 10:35:41.442336 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.124@KRBTEST.COM" created.
I20260929 10:35:41.443404 29160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:41.431489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:35179 (local address 127.0.0.1:50554)
0929 10:35:41.431911 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:41.431953 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 10:35:41.432826 (+ 873us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:35:41.433363 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:35:41.433370 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:35:41.433706 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:35:41.434172 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:41.434184 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:41.435037 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:41.435041 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:41.435905 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:41.435913 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:41.436437 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:41.436441 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:41.436623 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:41.437215 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:35:41.437232 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:35:41.438688 (+ 1456us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:35:41.440505 (+ 1817us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:35:41.440511 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:35:41.440527 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:35:41.440809 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:35:41.441093 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:35:41.441098 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:35:41.441100 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:35:41.441240 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:35:41.441680 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:35:41.441688 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:35:41.441951 (+ 263us) client_negotiation.cc:769] Sending connection context
0929 10:35:41.442137 (+ 186us) client_negotiation.cc:240] Negotiation successful
0929 10:35:41.442495 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":105,"threads_started":1}
I20260929 10:35:41.444996 29158 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.82.125" port: 44731 } has no permanent_uuid. Determining permanent_uuid...
I20260929 10:35:41.450918 29083 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.82.125" port: 44731 } attempt: 3
Sep 29 10:35:41 dist-test-slave-vg89 krb5kdc[29002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678141, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.125@KRBTEST.COM for kudu/127.15.82.125@KRBTEST.COM
I20260929 10:35:41.455785 29160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:41.445511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.125:44731 (local address 127.0.0.1:47344)
0929 10:35:41.445645 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:41.445664 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 10:35:41.445741 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:35:41.446487 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:35:41.446490 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:35:41.446632 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:35:41.446722 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:41.446727 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:41.448218 (+ 1491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:41.448220 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:41.448752 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:41.448759 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:41.449673 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:41.449676 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:41.449752 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:41.450245 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:35:41.450255 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:35:41.451635 (+ 1380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:35:41.454064 (+ 2429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:35:41.454067 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:35:41.454069 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:35:41.454249 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:35:41.454593 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:35:41.454597 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:35:41.454599 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:35:41.454695 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:35:41.455247 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:35:41.455249 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:35:41.455325 (+ 76us) client_negotiation.cc:769] Sending connection context
0929 10:35:41.455459 (+ 134us) client_negotiation.cc:240] Negotiation successful
0929 10:35:41.455599 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50,"mutex_wait_us":1}
I20260929 10:35:41.456069 29163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:41.445697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47344 (local address 127.15.82.125:44731)
0929 10:35:41.446028 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0929 10:35:41.446040 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 10:35:41.446065 (+ 25us) server_negotiation.cc:407] Connection header received
0929 10:35:41.446111 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:35:41.446118 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:35:41.446325 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:35:41.446491 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:35:41.446832 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:41.448108 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:41.448868 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:41.449550 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:41.449665 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:41.451793 (+ 2128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:35:41.451801 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:35:41.451815 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:35:41.451837 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:35:41.453941 (+ 2104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:35:41.454358 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:35:41.454364 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:35:41.454366 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:35:41.454503 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:35:41.454804 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:35:41.454810 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:35:41.454815 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:35:41.455141 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:35:41.455270 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0929 10:35:41.455542 (+ 272us) server_negotiation.cc:299] Negotiation successful
0929 10:35:41.455666 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":110,"threads_started":1}
Sep 29 10:35:41 dist-test-slave-vg89 krb5kdc[29002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678141, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.126@KRBTEST.COM for kudu/127.15.82.125@KRBTEST.COM
I20260929 10:35:41.458220 29158 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.82.124" port: 34231 } has no permanent_uuid. Determining permanent_uuid...
I20260929 10:35:41.458959 29160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:41.458712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.124:34231 (local address 127.0.0.1:34430)
0929 10:35:41.458820 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:41.458898 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.124:34231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260929 10:35:41.459196 29106 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.82.124:34231: connect: Connection refused (error 111)
I20260929 10:35:41.460125 29160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:41.459916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.124:34231 (local address 127.0.0.1:34442)
0929 10:35:41.459994 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:41.460036 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.124:34231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260929 10:35:41.460322 29158 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.82.124:34231: Network error: Client connection negotiation failed: client connection to 127.15.82.124:34231: connect: Connection refused (error 111)
I20260929 10:35:41.461272 29086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:41.451368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.125:44731 (local address 127.0.0.1:47358)
0929 10:35:41.451804 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:41.451819 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 10:35:41.451903 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:35:41.452230 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:35:41.452233 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:35:41.452407 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:35:41.452504 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:41.452509 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:41.453616 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:41.453619 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:41.454910 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:41.454919 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:41.455533 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:41.455537 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:41.455647 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:41.456244 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:35:41.456262 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:35:41.457692 (+ 1430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:35:41.459751 (+ 2059us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:35:41.459754 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:35:41.459757 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:35:41.460006 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:35:41.460365 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:35:41.460368 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:35:41.460370 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:35:41.460466 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:35:41.460872 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:35:41.460875 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:35:41.460968 (+ 93us) client_negotiation.cc:769] Sending connection context
0929 10:35:41.461056 (+ 88us) client_negotiation.cc:240] Negotiation successful
0929 10:35:41.461162 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343}
I20260929 10:35:41.461344 29164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:41.451453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47358 (local address 127.15.82.125:44731)
0929 10:35:41.451853 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0929 10:35:41.451862 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 10:35:41.451877 (+ 15us) server_negotiation.cc:407] Connection header received
0929 10:35:41.452002 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:35:41.452005 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:35:41.452055 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:35:41.452214 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:35:41.452625 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:41.453508 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:41.455013 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:41.455376 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:41.455525 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:41.457834 (+ 2309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:35:41.457838 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:35:41.457839 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:35:41.457882 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:35:41.459644 (+ 1762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:35:41.460142 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:35:41.460146 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:35:41.460149 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:35:41.460238 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:35:41.460554 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:35:41.460557 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:35:41.460561 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:35:41.460770 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:35:41.460860 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0929 10:35:41.461117 (+ 257us) server_negotiation.cc:299] Negotiation successful
0929 10:35:41.461207 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":179,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.124.keytab.
Entry for principal kudu/127.15.82.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.124.keytab.
I20260929 10:35:41.461979 29083 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.82.124" port: 34231 } has no permanent_uuid. Determining permanent_uuid...
I20260929 10:35:41.462592 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790677981592896-15689-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790677981592896-15689-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790677981592896-15689-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790677981592896-15689-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.124:34231
--webserver_interface=127.15.82.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.124.keytab
--principal=kudu/127.15.82.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:43761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.82.126:35179,127.15.82.125:44731,127.15.82.124:34231
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 10:35:41.462656 29086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:41.462456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.124:34231 (local address 127.0.0.1:34446)
0929 10:35:41.462543 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:41.462590 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.124:34231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260929 10:35:41.463446 29086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:41.463127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.124:34231 (local address 127.0.0.1:34454)
0929 10:35:41.463305 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:41.463371 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.124:34231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
W20260929 10:35:41.463584 29083 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.82.124:34231: Network error: Client connection negotiation failed: client connection to 127.15.82.124:34231: connect: Connection refused (error 111)
I20260929 10:35:41.494750 29083 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.82.124" port: 34231 } attempt: 1
I20260929 10:35:41.495728 29086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:41.495419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.124:34231 (local address 127.0.0.1:34462)
0929 10:35:41.495574 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:41.495642 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.124:34231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260929 10:35:41.496479 29086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:41.496282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.124:34231 (local address 127.0.0.1:34474)
0929 10:35:41.496374 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:41.496418 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.124:34231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260929 10:35:41.496676 29083 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.82.124:34231: Network error: Client connection negotiation failed: client connection to 127.15.82.124:34231: connect: Connection refused (error 111)
I20260929 10:35:41.509598 29158 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.82.124" port: 34231 } attempt: 1
I20260929 10:35:41.510560 29160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:41.510170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.124:34231 (local address 127.0.0.1:34478)
0929 10:35:41.510342 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:41.510462 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.124:34231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260929 10:35:41.511338 29160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:41.511120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.124:34231 (local address 127.0.0.1:34488)
0929 10:35:41.511228 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:41.511282 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.124:34231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260929 10:35:41.511467 29158 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.82.124:34231: Network error: Client connection negotiation failed: client connection to 127.15.82.124:34231: connect: Connection refused (error 111)
I20260929 10:35:41.571856 29083 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.82.124" port: 34231 } attempt: 2
I20260929 10:35:41.572717 29086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:41.572397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.124:34231 (local address 127.0.0.1:34492)
0929 10:35:41.572566 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:41.572635 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.124:34231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260929 10:35:41.573457 29086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:41.573241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.124:34231 (local address 127.0.0.1:34496)
0929 10:35:41.573350 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:41.573402 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.124:34231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260929 10:35:41.573599 29083 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.82.124:34231: Network error: Client connection negotiation failed: client connection to 127.15.82.124:34231: connect: Connection refused (error 111)
I20260929 10:35:41.579627 29158 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.82.124" port: 34231 } attempt: 2
I20260929 10:35:41.580376 29160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:41.580101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.124:34231 (local address 127.0.0.1:34506)
0929 10:35:41.580235 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:41.580304 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.124:34231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260929 10:35:41.581068 29160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:41.580859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.124:34231 (local address 127.0.0.1:34512)
0929 10:35:41.580959 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:41.581005 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.124:34231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260929 10:35:41.581194 29158 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.82.124:34231: Network error: Client connection negotiation failed: client connection to 127.15.82.124:34231: connect: Connection refused (error 111)
W20260929 10:35:41.583971 29166 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:35:41.584285 29166 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:35:41.584388 29166 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:35:41.589228 29166 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:35:41.589322 29166 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:35:41.589361 29166 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:35:41.589396 29166 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:35:41.589430 29166 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:35:41.595238 29166 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:43761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790677981592896-15689-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790677981592896-15689-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.15.82.126:35179,127.15.82.125:44731,127.15.82.124:34231
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.124.keytab
--principal=kudu/127.15.82.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.124:34231
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790677981592896-15689-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.15.82.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29166
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790677981592896-15689-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:35:41.596444 29166 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:35:41.597570 29166 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:35:41.606446 29172 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:35:41.606439 29171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:35:41.606472 29174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:35:41.606933 29166 server_base.cc:1034] running on GCE node
I20260929 10:35:41.607543 29166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:35:41.608793 29166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:35:41.609957 29166 hybrid_clock.cc:648] HybridClock initialized: now 1790678141609936 us; error 42 us; skew 500 ppm
Sep 29 10:35:41 dist-test-slave-vg89 krb5kdc[29002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678141, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:35:41.612911 29166 init.cc:377] Logged in from keytab as kudu/127.15.82.124@KRBTEST.COM (short username kudu)
I20260929 10:35:41.613830 29166 webserver.cc:460] Webserver started at http://127.15.82.124:38077/ using document root <none> and password file <none>
I20260929 10:35:41.614387 29166 fs_manager.cc:359] Metadata directory not provided
I20260929 10:35:41.614526 29166 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:35:41.614714 29166 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:35:41.616782 29166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790677981592896-15689-0/minicluster-data/master-2/data/instance:
uuid: "85aa0a25b95441f68dec06be351a1674"
format_stamp: "Formatted at 2026-09-29 10:35:41 on dist-test-slave-vg89"
server_key: "ae29d0c463ea929fe1adce7206c11310"
server_key_iv: "1f107841270248b1327cbdc80553b629"
server_key_version: "encryptionkey@0"
I20260929 10:35:41.617316 29166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790677981592896-15689-0/minicluster-data/master-2/wal/instance:
uuid: "85aa0a25b95441f68dec06be351a1674"
format_stamp: "Formatted at 2026-09-29 10:35:41 on dist-test-slave-vg89"
server_key: "ae29d0c463ea929fe1adce7206c11310"
server_key_iv: "1f107841270248b1327cbdc80553b629"
server_key_version: "encryptionkey@0"
I20260929 10:35:41.621279 29166 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260929 10:35:41.623979 29181 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:35:41.625335 29166 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 10:35:41.625464 29166 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790677981592896-15689-0/minicluster-data/master-2/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790677981592896-15689-0/minicluster-data/master-2/wal
uuid: "85aa0a25b95441f68dec06be351a1674"
format_stamp: "Formatted at 2026-09-29 10:35:41 on dist-test-slave-vg89"
server_key: "ae29d0c463ea929fe1adce7206c11310"
server_key_iv: "1f107841270248b1327cbdc80553b629"
server_key_version: "encryptionkey@0"
I20260929 10:35:41.625566 29166 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790677981592896-15689-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790677981592896-15689-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790677981592896-15689-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:35:41.644038 29166 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:35:41.648069 29166 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:35:41.648257 29166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:35:41.656250 29166 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.124:34231
I20260929 10:35:41.656271 29233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.124:34231 every 8 connection(s)
I20260929 10:35:41.657400 29166 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790677981592896-15689-0/minicluster-data/master-2/data/info.pb
I20260929 10:35:41.659695 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 29166
I20260929 10:35:41.659804 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790677981592896-15689-0/minicluster-data/master-2/wal/instance
I20260929 10:35:41.660127 15689 external_mini_cluster.cc:1428] Setting key 8403faee49c0b8b5cb87e4582ceb393a
I20260929 10:35:41.660940 29234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:35:41.665941 29234 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.82.126" port: 35179 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 10:35:41 dist-test-slave-vg89 krb5kdc[29002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678140, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.82.124@KRBTEST.COM
I20260929 10:35:41.672374 29158 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.82.124" port: 34231 } attempt: 3
I20260929 10:35:41.672781 29083 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.82.124" port: 34231 } attempt: 3
Sep 29 10:35:41 dist-test-slave-vg89 krb5kdc[29002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678141, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.124@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:35:41.679139 29237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:41.661694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34528 (local address 127.15.82.124:34231)
0929 10:35:41.662142 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0929 10:35:41.662163 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0929 10:35:41.662208 (+ 45us) server_negotiation.cc:407] Connection header received
0929 10:35:41.663146 (+ 938us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:35:41.663172 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:35:41.663635 (+ 463us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:35:41.664054 (+ 419us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:35:41.664846 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:41.666354 (+ 1508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:41.667562 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:41.668409 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:41.668590 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:41.671287 (+ 2697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:35:41.671293 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:35:41.671306 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:35:41.671340 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:35:41.674232 (+ 2892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:35:41.674887 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:35:41.674893 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:35:41.674901 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:35:41.675046 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:35:41.675412 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:35:41.675415 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:35:41.675417 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:35:41.676047 (+ 630us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:35:41.677251 (+ 1204us) server_negotiation.cc:1035] Waiting for connection context
0929 10:35:41.677454 (+ 203us) server_negotiation.cc:299] Negotiation successful
0929 10:35:41.677794 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":123,"threads_started":1}
Sep 29 10:35:41 dist-test-slave-vg89 krb5kdc[29002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678141, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.126@KRBTEST.COM for kudu/127.15.82.124@KRBTEST.COM
Sep 29 10:35:41 dist-test-slave-vg89 krb5kdc[29002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678141, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.125@KRBTEST.COM for kudu/127.15.82.124@KRBTEST.COM
I20260929 10:35:41.679993 29087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:41.668161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50564 (local address 127.15.82.126:35179)
0929 10:35:41.668296 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0929 10:35:41.668301 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:35:41.668658 (+ 357us) server_negotiation.cc:407] Connection header received
0929 10:35:41.668887 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:35:41.668890 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:35:41.668939 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:35:41.669046 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:35:41.669808 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:41.670345 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:41.671552 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:41.671881 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:41.672041 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:41.675405 (+ 3364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:35:41.675410 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:35:41.675413 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:35:41.675439 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:35:41.677319 (+ 1880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:35:41.677876 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:35:41.677880 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:35:41.677882 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:35:41.678008 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:35:41.678975 (+ 967us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:35:41.678978 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:35:41.678980 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:35:41.679172 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:35:41.679289 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0929 10:35:41.679702 (+ 413us) server_negotiation.cc:299] Negotiation successful
0929 10:35:41.679819 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260929 10:35:41.680006 29238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:41.668073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:35179 (local address 127.0.0.1:50564)
0929 10:35:41.668527 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:41.668581 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0929 10:35:41.668754 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:35:41.669131 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:35:41.669140 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:35:41.669534 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:35:41.669674 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:41.669680 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:41.670519 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:41.670524 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:41.671412 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:41.671425 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:41.672056 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:41.672060 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:41.672214 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:41.673277 (+ 1063us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:35:41.673296 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:35:41.675223 (+ 1927us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:35:41.677446 (+ 2223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:35:41.677454 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:35:41.677460 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:35:41.677746 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:35:41.678135 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:35:41.678139 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:35:41.678141 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:35:41.678858 (+ 717us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:35:41.679353 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:35:41.679360 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:35:41.679483 (+ 123us) client_negotiation.cc:769] Sending connection context
0929 10:35:41.679644 (+ 161us) client_negotiation.cc:240] Negotiation successful
0929 10:35:41.679819 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"spinlock_wait_cycles":1342336,"thread_start_us":117,"threads_started":1}
I20260929 10:35:41.681425 29234 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.82.125" port: 44731 } has no permanent_uuid. Determining permanent_uuid...
I20260929 10:35:41.685720 29086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:41.673195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.124:34231 (local address 127.0.0.1:34536)
0929 10:35:41.673376 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:41.673390 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 10:35:41.673468 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:35:41.674232 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:35:41.674234 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:35:41.674458 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:35:41.674550 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:41.674557 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:41.676148 (+ 1591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:41.676152 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:41.676960 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:41.676970 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:41.677547 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:41.677553 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:41.677660 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:41.678128 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:35:41.678143 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:35:41.680366 (+ 2223us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:35:41.683928 (+ 3562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:35:41.683934 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:35:41.683937 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:35:41.684229 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:35:41.684548 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:35:41.684551 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:35:41.684554 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:35:41.684657 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:35:41.685181 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:35:41.685184 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:35:41.685292 (+ 108us) client_negotiation.cc:769] Sending connection context
0929 10:35:41.685385 (+ 93us) client_negotiation.cc:240] Negotiation successful
0929 10:35:41.685531 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109}
I20260929 10:35:41.686539 29240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:41.673649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34536 (local address 127.15.82.124:34231)
0929 10:35:41.674002 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0929 10:35:41.674011 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 10:35:41.674023 (+ 12us) server_negotiation.cc:407] Connection header received
0929 10:35:41.674054 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:35:41.674057 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:35:41.674091 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:35:41.674666 (+ 575us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:35:41.674760 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:41.676025 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:41.677078 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:41.677425 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:41.677864 (+ 439us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:41.680926 (+ 3062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:35:41.680929 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:35:41.680932 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:35:41.680952 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:35:41.683787 (+ 2835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:35:41.684349 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:35:41.684351 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:35:41.684353 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:35:41.684427 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:35:41.684802 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:35:41.684806 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:35:41.684808 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:35:41.685032 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:35:41.686061 (+ 1029us) server_negotiation.cc:1035] Waiting for connection context
0929 10:35:41.686141 (+ 80us) server_negotiation.cc:299] Negotiation successful
0929 10:35:41.686257 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":147,"threads_started":1}
I20260929 10:35:41.686560 29160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:41.673050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.124:34231 (local address 127.0.0.1:34530)
0929 10:35:41.673200 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:41.673217 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 10:35:41.673300 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:35:41.673768 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:35:41.673774 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:35:41.673994 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:35:41.674118 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:41.674128 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:41.675385 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:41.675389 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:41.676779 (+ 1390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:41.676791 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:41.677372 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:41.677376 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:41.677470 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:41.678148 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:35:41.678162 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:35:41.680781 (+ 2619us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:35:41.684927 (+ 4146us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:35:41.684931 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:35:41.684933 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:35:41.685211 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:35:41.685538 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:35:41.685542 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:35:41.685544 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:35:41.685637 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:35:41.686028 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:35:41.686031 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:35:41.686136 (+ 105us) client_negotiation.cc:769] Sending connection context
0929 10:35:41.686218 (+ 82us) client_negotiation.cc:240] Negotiation successful
0929 10:35:41.686350 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260929 10:35:41.687443 29239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:41.673030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34530 (local address 127.15.82.124:34231)
0929 10:35:41.673522 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0929 10:35:41.673533 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 10:35:41.673547 (+ 14us) server_negotiation.cc:407] Connection header received
0929 10:35:41.673580 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:35:41.673585 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:35:41.673628 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:35:41.674228 (+ 600us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:35:41.674322 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:41.675248 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:41.676930 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:41.677251 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:41.678042 (+ 791us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:41.680902 (+ 2860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:35:41.680906 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:35:41.680909 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:35:41.680933 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:35:41.684813 (+ 3880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:35:41.685329 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:35:41.685332 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:35:41.685334 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:35:41.685430 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:35:41.685740 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:35:41.685743 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:35:41.685745 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:35:41.685921 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:35:41.687011 (+ 1090us) server_negotiation.cc:1035] Waiting for connection context
0929 10:35:41.687095 (+ 84us) server_negotiation.cc:299] Negotiation successful
0929 10:35:41.687212 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":97,"threads_started":1}
Sep 29 10:35:41 dist-test-slave-vg89 krb5kdc[29002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678141, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.124@KRBTEST.COM for kudu/127.15.82.125@KRBTEST.COM
Sep 29 10:35:41 dist-test-slave-vg89 krb5kdc[29002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678140, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:35:41.690465 29083 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f2e41826e584a2e8864dd419989f112: Bootstrap starting.
I20260929 10:35:41.693593 29083 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f2e41826e584a2e8864dd419989f112: Neither blocks nor log segments found. Creating new log.
I20260929 10:35:41.694124 29164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:41.682838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47372 (local address 127.15.82.125:44731)
0929 10:35:41.682970 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0929 10:35:41.682986 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0929 10:35:41.683003 (+ 17us) server_negotiation.cc:407] Connection header received
0929 10:35:41.683050 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:35:41.683053 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:35:41.683104 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:35:41.683187 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:35:41.684218 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:41.685008 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:41.686512 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:41.686912 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:41.686993 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:41.689924 (+ 2931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:35:41.689928 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:35:41.689932 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:35:41.689960 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:35:41.691836 (+ 1876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:35:41.692586 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:35:41.692591 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:35:41.692596 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:35:41.692695 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:35:41.693032 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:35:41.693035 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:35:41.693037 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:35:41.693374 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:35:41.693489 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0929 10:35:41.693823 (+ 334us) server_negotiation.cc:299] Negotiation successful
0929 10:35:41.693974 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260929 10:35:41.694645 29238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:41.682592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.125:44731 (local address 127.0.0.1:47372)
0929 10:35:41.682841 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:41.682854 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 10:35:41.682936 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:35:41.683778 (+ 842us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:35:41.683781 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:35:41.683954 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:35:41.684080 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:41.684087 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:41.685126 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:41.685133 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:41.686307 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:41.686317 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:41.687763 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:41.687767 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:41.687839 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:41.688242 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:35:41.688253 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:35:41.689752 (+ 1499us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:35:41.691968 (+ 2216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:35:41.691972 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:35:41.691976 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:35:41.692468 (+ 492us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:35:41.692808 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:35:41.692811 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:35:41.692813 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:35:41.692918 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:35:41.693497 (+ 579us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:35:41.693503 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:35:41.693613 (+ 110us) client_negotiation.cc:769] Sending connection context
0929 10:35:41.694324 (+ 711us) client_negotiation.cc:240] Negotiation successful
0929 10:35:41.694501 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177}
I20260929 10:35:41.695484 29087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:41.682592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50566 (local address 127.15.82.126:35179)
0929 10:35:41.682703 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0929 10:35:41.682708 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:35:41.682898 (+ 190us) server_negotiation.cc:407] Connection header received
0929 10:35:41.683042 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:35:41.683044 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:35:41.683092 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:35:41.683180 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:35:41.683806 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:41.684413 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:41.686173 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:41.686533 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:41.686631 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:41.691209 (+ 4578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:35:41.691214 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:35:41.691218 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:35:41.691250 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:35:41.692933 (+ 1683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:35:41.693528 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:35:41.693532 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:35:41.693533 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:35:41.693625 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:35:41.693964 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:35:41.693968 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:35:41.693970 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:35:41.694192 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:35:41.694391 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0929 10:35:41.695191 (+ 800us) server_negotiation.cc:299] Negotiation successful
0929 10:35:41.695322 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260929 10:35:41.695567 29234 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.82.124" port: 34231 } has no permanent_uuid. Determining permanent_uuid...
I20260929 10:35:41.696882 29158 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b222a401dd004f67b25de657fcd29257: Bootstrap starting.
I20260929 10:35:41.699596 29158 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b222a401dd004f67b25de657fcd29257: Neither blocks nor log segments found. Creating new log.
I20260929 10:35:41.700343 29158 log.cc:824] T 00000000000000000000000000000000 P b222a401dd004f67b25de657fcd29257: Log is configured to *not* fsync() on all Append() calls
Sep 29 10:35:41 dist-test-slave-vg89 krb5kdc[29002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678141, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.124@KRBTEST.COM for kudu/127.15.82.124@KRBTEST.COM
I20260929 10:35:41.701236 29083 log.cc:824] T 00000000000000000000000000000000 P 6f2e41826e584a2e8864dd419989f112: Log is configured to *not* fsync() on all Append() calls
I20260929 10:35:41.703044 29158 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b222a401dd004f67b25de657fcd29257: No bootstrap required, opened a new log
I20260929 10:35:41.703332 29083 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f2e41826e584a2e8864dd419989f112: No bootstrap required, opened a new log
I20260929 10:35:41.705291 29240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:41.696184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34546 (local address 127.15.82.124:34231)
0929 10:35:41.696278 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0929 10:35:41.696282 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:35:41.696316 (+ 34us) server_negotiation.cc:407] Connection header received
0929 10:35:41.696349 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:35:41.696351 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:35:41.696391 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:35:41.696465 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:35:41.696824 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:41.697581 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:41.698541 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:41.698879 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:41.698995 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:41.701069 (+ 2074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:35:41.701074 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:35:41.701076 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:35:41.701100 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:35:41.703023 (+ 1923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:35:41.703847 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:35:41.703854 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:35:41.703859 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:35:41.703946 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:35:41.704296 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:35:41.704301 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:35:41.704303 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:35:41.704524 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:35:41.704610 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0929 10:35:41.704962 (+ 352us) server_negotiation.cc:299] Negotiation successful
0929 10:35:41.705103 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260929 10:35:41.705749 29238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:41.696061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.124:34231 (local address 127.0.0.1:34546)
0929 10:35:41.696169 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:41.696184 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 10:35:41.696261 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:35:41.696478 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:35:41.696481 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:35:41.696619 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:35:41.696706 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:41.696710 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:41.697668 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:41.697671 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:41.698290 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:41.698300 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:41.699007 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:41.699010 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:41.699075 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:41.699556 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:35:41.699568 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:35:41.700924 (+ 1356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:35:41.703467 (+ 2543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:35:41.703470 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:35:41.703472 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:35:41.703744 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:35:41.704081 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:35:41.704088 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:35:41.704092 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:35:41.704187 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:35:41.704693 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:35:41.704696 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:35:41.704791 (+ 95us) client_negotiation.cc:769] Sending connection context
0929 10:35:41.705423 (+ 632us) client_negotiation.cc:240] Negotiation successful
0929 10:35:41.705551 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260929 10:35:41.707360 29083 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f2e41826e584a2e8864dd419989f112 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f2e41826e584a2e8864dd419989f112" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 35179 } } peers { permanent_uuid: "b222a401dd004f67b25de657fcd29257" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 44731 } } peers { permanent_uuid: "85aa0a25b95441f68dec06be351a1674" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 34231 } }
I20260929 10:35:41.707643 29083 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6f2e41826e584a2e8864dd419989f112 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:35:41.707703 29083 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f2e41826e584a2e8864dd419989f112 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f2e41826e584a2e8864dd419989f112, State: Initialized, Role: FOLLOWER
I20260929 10:35:41.708194 29083 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f2e41826e584a2e8864dd419989f112 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f2e41826e584a2e8864dd419989f112" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 35179 } } peers { permanent_uuid: "b222a401dd004f67b25de657fcd29257" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 44731 } } peers { permanent_uuid: "85aa0a25b95441f68dec06be351a1674" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 34231 } }
I20260929 10:35:41.708853 29234 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85aa0a25b95441f68dec06be351a1674: Bootstrap starting.
I20260929 10:35:41.708772 29243 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f2e41826e584a2e8864dd419989f112 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f2e41826e584a2e8864dd419989f112" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 35179 } } peers { permanent_uuid: "b222a401dd004f67b25de657fcd29257" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 44731 } } peers { permanent_uuid: "85aa0a25b95441f68dec06be351a1674" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 34231 } } }
I20260929 10:35:41.709084 29243 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f2e41826e584a2e8864dd419989f112 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 10:35:41.709615 29083 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f2e41826e584a2e8864dd419989f112 [sys.catalog]: configured and running, proceeding with master startup.
Sep 29 10:35:41 dist-test-slave-vg89 krb5kdc[29002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678140, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.82.125@KRBTEST.COM
I20260929 10:35:41.711809 29234 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 85aa0a25b95441f68dec06be351a1674: Neither blocks nor log segments found. Creating new log.
I20260929 10:35:41.712605 29234 log.cc:824] T 00000000000000000000000000000000 P 85aa0a25b95441f68dec06be351a1674: Log is configured to *not* fsync() on all Append() calls
I20260929 10:35:41.714962 29164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:41.702971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47386 (local address 127.15.82.125:44731)
0929 10:35:41.703113 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0929 10:35:41.703119 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 10:35:41.703302 (+ 183us) server_negotiation.cc:407] Connection header received
0929 10:35:41.703485 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:35:41.703488 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:35:41.703545 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:35:41.703632 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:35:41.704701 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:41.705558 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:41.706999 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:41.707252 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:41.707377 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:41.710527 (+ 3150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:35:41.710533 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:35:41.710536 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:35:41.710570 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:35:41.712410 (+ 1840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:35:41.712997 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:35:41.713001 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:35:41.713005 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:35:41.713106 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:35:41.713452 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:35:41.713456 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:35:41.713458 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:35:41.713711 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:35:41.713930 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0929 10:35:41.714668 (+ 738us) server_negotiation.cc:299] Negotiation successful
0929 10:35:41.714799 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260929 10:35:41.715240 29234 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85aa0a25b95441f68dec06be351a1674: No bootstrap required, opened a new log
I20260929 10:35:41.718377 15689 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 10:35:41.719736 29234 raft_consensus.cc:348] T 00000000000000000000000000000000 P 85aa0a25b95441f68dec06be351a1674 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f2e41826e584a2e8864dd419989f112" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 35179 } } peers { permanent_uuid: "b222a401dd004f67b25de657fcd29257" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 44731 } } peers { permanent_uuid: "85aa0a25b95441f68dec06be351a1674" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 34231 } }
I20260929 10:35:41.720026 29234 raft_consensus.cc:374] T 00000000000000000000000000000000 P 85aa0a25b95441f68dec06be351a1674 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:35:41.720085 29234 raft_consensus.cc:729] T 00000000000000000000000000000000 P 85aa0a25b95441f68dec06be351a1674 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85aa0a25b95441f68dec06be351a1674, State: Initialized, Role: FOLLOWER
I20260929 10:35:41.715778 29015 master_runner.cc:186] Error getting master registration for 127.15.82.125:44731: OK, instance_id { permanent_uuid: "b222a401dd004f67b25de657fcd29257" instance_seqno: 1790678141419848 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 10:35:41.720301 29015 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 10:35:41.720543 29234 consensus_queue.cc:260] T 00000000000000000000000000000000 P 85aa0a25b95441f68dec06be351a1674 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f2e41826e584a2e8864dd419989f112" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 35179 } } peers { permanent_uuid: "b222a401dd004f67b25de657fcd29257" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 44731 } } peers { permanent_uuid: "85aa0a25b95441f68dec06be351a1674" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 34231 } }
W20260929 10:35:41.721870 29254 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6f2e41826e584a2e8864dd419989f112: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 10:35:41.721967 29254 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 10:35:41.721985 29234 sys_catalog.cc:564] T 00000000000000000000000000000000 P 85aa0a25b95441f68dec06be351a1674 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:35:41.722651 29261 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85aa0a25b95441f68dec06be351a1674 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f2e41826e584a2e8864dd419989f112" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 35179 } } peers { permanent_uuid: "b222a401dd004f67b25de657fcd29257" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 44731 } } peers { permanent_uuid: "85aa0a25b95441f68dec06be351a1674" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 34231 } } }
I20260929 10:35:41.722780 29261 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85aa0a25b95441f68dec06be351a1674 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 10:35:41.723522 29158 raft_consensus.cc:348] T 00000000000000000000000000000000 P b222a401dd004f67b25de657fcd29257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f2e41826e584a2e8864dd419989f112" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 35179 } } peers { permanent_uuid: "b222a401dd004f67b25de657fcd29257" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 44731 } } peers { permanent_uuid: "85aa0a25b95441f68dec06be351a1674" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 34231 } }
I20260929 10:35:41.723788 29158 raft_consensus.cc:374] T 00000000000000000000000000000000 P b222a401dd004f67b25de657fcd29257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:35:41.723857 29158 raft_consensus.cc:729] T 00000000000000000000000000000000 P b222a401dd004f67b25de657fcd29257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b222a401dd004f67b25de657fcd29257, State: Initialized, Role: FOLLOWER
I20260929 10:35:41.724346 29158 consensus_queue.cc:260] T 00000000000000000000000000000000 P b222a401dd004f67b25de657fcd29257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f2e41826e584a2e8864dd419989f112" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 35179 } } peers { permanent_uuid: "b222a401dd004f67b25de657fcd29257" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 44731 } } peers { permanent_uuid: "85aa0a25b95441f68dec06be351a1674" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 34231 } }
I20260929 10:35:41.724858 29265 sys_catalog.cc:455] T 00000000000000000000000000000000 P b222a401dd004f67b25de657fcd29257 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f2e41826e584a2e8864dd419989f112" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 35179 } } peers { permanent_uuid: "b222a401dd004f67b25de657fcd29257" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 44731 } } peers { permanent_uuid: "85aa0a25b95441f68dec06be351a1674" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 34231 } } }
I20260929 10:35:41.725108 29265 sys_catalog.cc:458] T 00000000000000000000000000000000 P b222a401dd004f67b25de657fcd29257 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 10:35:41.725718 29158 sys_catalog.cc:564] T 00000000000000000000000000000000 P b222a401dd004f67b25de657fcd29257 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:35:41.733265 29240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:41.720941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34556 (local address 127.15.82.124:34231)
0929 10:35:41.721082 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0929 10:35:41.721087 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:35:41.723100 (+ 2013us) server_negotiation.cc:407] Connection header received
0929 10:35:41.723216 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:35:41.723218 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:35:41.723265 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:35:41.723375 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:35:41.724318 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:41.725137 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:41.726148 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:41.726483 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:41.726588 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:41.728466 (+ 1878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:35:41.728470 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:35:41.728492 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:35:41.728522 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:35:41.730526 (+ 2004us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:35:41.731349 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:35:41.731355 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:35:41.731359 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:35:41.731463 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:35:41.731847 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:35:41.731854 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:35:41.731856 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:35:41.732130 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:35:41.732288 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0929 10:35:41.732975 (+ 687us) server_negotiation.cc:299] Negotiation successful
0929 10:35:41.733102 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
W20260929 10:35:41.733794 29276 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b222a401dd004f67b25de657fcd29257: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 10:35:41.733953 29276 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260929 10:35:41.739455 29287 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 85aa0a25b95441f68dec06be351a1674: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 10:35:41.739542 29287 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 10:35:41.741183 29166 master_runner.cc:263] No leader master found from master 85aa0a25b95441f68dec06be351a1674
I20260929 10:35:41.741266 29166 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 10:35:41.741843 29164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:41.721577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47396 (local address 127.15.82.125:44731)
0929 10:35:41.721695 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0929 10:35:41.721699 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:35:41.723653 (+ 1954us) server_negotiation.cc:407] Connection header received
0929 10:35:41.728975 (+ 5322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:35:41.728979 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:35:41.729031 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:35:41.729099 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:35:41.731156 (+ 2057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:41.731826 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:41.735214 (+ 3388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:41.735544 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:41.735623 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:41.738068 (+ 2445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:35:41.738072 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:35:41.738075 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:35:41.738100 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:35:41.739864 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:35:41.740362 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:35:41.740366 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:35:41.740368 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:35:41.740473 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:35:41.740791 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:35:41.740795 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:35:41.740797 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:35:41.741029 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:35:41.741145 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0929 10:35:41.741554 (+ 409us) server_negotiation.cc:299] Negotiation successful
0929 10:35:41.741665 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260929 10:35:41.742261 29090 master_runner.cc:263] No leader master found from master b222a401dd004f67b25de657fcd29257
I20260929 10:35:41.742319 29090 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 10:35:41.746833 29087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:41.720894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50574 (local address 127.15.82.126:35179)
0929 10:35:41.721011 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0929 10:35:41.721016 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:35:41.723444 (+ 2428us) server_negotiation.cc:407] Connection header received
0929 10:35:41.729260 (+ 5816us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:35:41.729263 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:35:41.729322 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:35:41.729375 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:35:41.731976 (+ 2601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:41.732674 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:41.736169 (+ 3495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:41.736480 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:41.736556 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:41.743023 (+ 6467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:35:41.743028 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:35:41.743031 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:35:41.743065 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:35:41.744852 (+ 1787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:35:41.745363 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:35:41.745370 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:35:41.745374 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:35:41.745460 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:35:41.745767 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:35:41.745771 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:35:41.745773 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:35:41.746010 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:35:41.746132 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0929 10:35:41.746537 (+ 405us) server_negotiation.cc:299] Negotiation successful
0929 10:35:41.746668 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
W20260929 10:35:41.899734 29187 tablet.cc:2369] T 00000000000000000000000000000000 P 85aa0a25b95441f68dec06be351a1674: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 10:35:41.916321 29111 tablet.cc:2369] T 00000000000000000000000000000000 P b222a401dd004f67b25de657fcd29257: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 10:35:41.952800 29036 tablet.cc:2369] T 00000000000000000000000000000000 P 6f2e41826e584a2e8864dd419989f112: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 10:35:41.961164 29265 raft_consensus.cc:482] T 00000000000000000000000000000000 P b222a401dd004f67b25de657fcd29257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:35:41.961359 29265 raft_consensus.cc:504] T 00000000000000000000000000000000 P b222a401dd004f67b25de657fcd29257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f2e41826e584a2e8864dd419989f112" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 35179 } } peers { permanent_uuid: "b222a401dd004f67b25de657fcd29257" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 44731 } } peers { permanent_uuid: "85aa0a25b95441f68dec06be351a1674" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 34231 } }
I20260929 10:35:41.962231 29265 leader_election.cc:288] T 00000000000000000000000000000000 P b222a401dd004f67b25de657fcd29257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f2e41826e584a2e8864dd419989f112 (127.15.82.126:35179), 85aa0a25b95441f68dec06be351a1674 (127.15.82.124:34231)
I20260929 10:35:41.962864 29208 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b222a401dd004f67b25de657fcd29257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85aa0a25b95441f68dec06be351a1674" is_pre_election: true
I20260929 10:35:41.962877 29057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b222a401dd004f67b25de657fcd29257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f2e41826e584a2e8864dd419989f112" is_pre_election: true
I20260929 10:35:41.963176 29057 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6f2e41826e584a2e8864dd419989f112 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b222a401dd004f67b25de657fcd29257 in term 0.
I20260929 10:35:41.963177 29208 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 85aa0a25b95441f68dec06be351a1674 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b222a401dd004f67b25de657fcd29257 in term 0.
I20260929 10:35:41.963683 29106 leader_election.cc:302] T 00000000000000000000000000000000 P b222a401dd004f67b25de657fcd29257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 6f2e41826e584a2e8864dd419989f112, 85aa0a25b95441f68dec06be351a1674, b222a401dd004f67b25de657fcd29257; no voters:
I20260929 10:35:41.964148 29265 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b222a401dd004f67b25de657fcd29257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:35:41.964260 29265 raft_consensus.cc:482] T 00000000000000000000000000000000 P b222a401dd004f67b25de657fcd29257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:35:41.964309 29265 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b222a401dd004f67b25de657fcd29257 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:35:41.965387 29265 raft_consensus.cc:504] T 00000000000000000000000000000000 P b222a401dd004f67b25de657fcd29257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f2e41826e584a2e8864dd419989f112" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 35179 } } peers { permanent_uuid: "b222a401dd004f67b25de657fcd29257" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 44731 } } peers { permanent_uuid: "85aa0a25b95441f68dec06be351a1674" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 34231 } }
I20260929 10:35:41.965786 29265 leader_election.cc:288] T 00000000000000000000000000000000 P b222a401dd004f67b25de657fcd29257 [CANDIDATE]: Term 1 election: Requested vote from peers 6f2e41826e584a2e8864dd419989f112 (127.15.82.126:35179), 85aa0a25b95441f68dec06be351a1674 (127.15.82.124:34231)
I20260929 10:35:41.966061 29057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b222a401dd004f67b25de657fcd29257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f2e41826e584a2e8864dd419989f112"
I20260929 10:35:41.966140 29208 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b222a401dd004f67b25de657fcd29257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85aa0a25b95441f68dec06be351a1674"
I20260929 10:35:41.966215 29057 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f2e41826e584a2e8864dd419989f112 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:35:41.966272 29208 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 85aa0a25b95441f68dec06be351a1674 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:35:41.967295 29057 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6f2e41826e584a2e8864dd419989f112 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b222a401dd004f67b25de657fcd29257 in term 1.
I20260929 10:35:41.967303 29208 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 85aa0a25b95441f68dec06be351a1674 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b222a401dd004f67b25de657fcd29257 in term 1.
I20260929 10:35:41.967643 29106 leader_election.cc:302] T 00000000000000000000000000000000 P b222a401dd004f67b25de657fcd29257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85aa0a25b95441f68dec06be351a1674, b222a401dd004f67b25de657fcd29257; no voters:
I20260929 10:35:41.967851 29265 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b222a401dd004f67b25de657fcd29257 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:35:41.968077 29265 raft_consensus.cc:686] T 00000000000000000000000000000000 P b222a401dd004f67b25de657fcd29257 [term 1 LEADER]: Becoming Leader. State: Replica: b222a401dd004f67b25de657fcd29257, State: Running, Role: LEADER
I20260929 10:35:41.968394 29265 consensus_queue.cc:237] T 00000000000000000000000000000000 P b222a401dd004f67b25de657fcd29257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f2e41826e584a2e8864dd419989f112" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 35179 } } peers { permanent_uuid: "b222a401dd004f67b25de657fcd29257" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 44731 } } peers { permanent_uuid: "85aa0a25b95441f68dec06be351a1674" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 34231 } }
I20260929 10:35:41.970175 29290 sys_catalog.cc:455] T 00000000000000000000000000000000 P b222a401dd004f67b25de657fcd29257 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b222a401dd004f67b25de657fcd29257. Latest consensus state: current_term: 1 leader_uuid: "b222a401dd004f67b25de657fcd29257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f2e41826e584a2e8864dd419989f112" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 35179 } } peers { permanent_uuid: "b222a401dd004f67b25de657fcd29257" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 44731 } } peers { permanent_uuid: "85aa0a25b95441f68dec06be351a1674" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 34231 } } }
I20260929 10:35:41.970304 29290 sys_catalog.cc:458] T 00000000000000000000000000000000 P b222a401dd004f67b25de657fcd29257 [sys.catalog]: This master's current role is: LEADER
I20260929 10:35:41.970681 29292 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:35:41.971482 29292 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:35:41.975973 29208 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 85aa0a25b95441f68dec06be351a1674 [term 1 FOLLOWER]: Refusing update from remote peer b222a401dd004f67b25de657fcd29257: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:35:41.975975 29057 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6f2e41826e584a2e8864dd419989f112 [term 1 FOLLOWER]: Refusing update from remote peer b222a401dd004f67b25de657fcd29257: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:35:41.976626 29290 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b222a401dd004f67b25de657fcd29257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85aa0a25b95441f68dec06be351a1674" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 34231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:35:41.976760 29265 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b222a401dd004f67b25de657fcd29257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f2e41826e584a2e8864dd419989f112" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 35179 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:35:41.980569 29261 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85aa0a25b95441f68dec06be351a1674 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b222a401dd004f67b25de657fcd29257. Latest consensus state: current_term: 1 leader_uuid: "b222a401dd004f67b25de657fcd29257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f2e41826e584a2e8864dd419989f112" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 35179 } } peers { permanent_uuid: "b222a401dd004f67b25de657fcd29257" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 44731 } } peers { permanent_uuid: "85aa0a25b95441f68dec06be351a1674" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 34231 } } }
I20260929 10:35:41.980569 29243 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f2e41826e584a2e8864dd419989f112 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b222a401dd004f67b25de657fcd29257. Latest consensus state: current_term: 1 leader_uuid: "b222a401dd004f67b25de657fcd29257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f2e41826e584a2e8864dd419989f112" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 35179 } } peers { permanent_uuid: "b222a401dd004f67b25de657fcd29257" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 44731 } } peers { permanent_uuid: "85aa0a25b95441f68dec06be351a1674" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 34231 } } }
I20260929 10:35:41.980695 29261 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85aa0a25b95441f68dec06be351a1674 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 10:35:41.980695 29243 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f2e41826e584a2e8864dd419989f112 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 10:35:41.983337 29265 sys_catalog.cc:455] T 00000000000000000000000000000000 P b222a401dd004f67b25de657fcd29257 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b222a401dd004f67b25de657fcd29257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f2e41826e584a2e8864dd419989f112" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 35179 } } peers { permanent_uuid: "b222a401dd004f67b25de657fcd29257" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 44731 } } peers { permanent_uuid: "85aa0a25b95441f68dec06be351a1674" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 34231 } } }
I20260929 10:35:41.983472 29265 sys_catalog.cc:458] T 00000000000000000000000000000000 P b222a401dd004f67b25de657fcd29257 [sys.catalog]: This master's current role is: LEADER
I20260929 10:35:41.983741 29290 sys_catalog.cc:455] T 00000000000000000000000000000000 P b222a401dd004f67b25de657fcd29257 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b222a401dd004f67b25de657fcd29257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f2e41826e584a2e8864dd419989f112" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 35179 } } peers { permanent_uuid: "b222a401dd004f67b25de657fcd29257" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 44731 } } peers { permanent_uuid: "85aa0a25b95441f68dec06be351a1674" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 34231 } } }
I20260929 10:35:41.983865 29290 sys_catalog.cc:458] T 00000000000000000000000000000000 P b222a401dd004f67b25de657fcd29257 [sys.catalog]: This master's current role is: LEADER
I20260929 10:35:41.985167 29292 catalog_manager.cc:1347] Generated new cluster ID: c129e473c5b84b9b88bf5d2c6e4466d5
I20260929 10:35:41.985232 29292 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:35:41.985250 29243 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f2e41826e584a2e8864dd419989f112 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b222a401dd004f67b25de657fcd29257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f2e41826e584a2e8864dd419989f112" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 35179 } } peers { permanent_uuid: "b222a401dd004f67b25de657fcd29257" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 44731 } } peers { permanent_uuid: "85aa0a25b95441f68dec06be351a1674" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 34231 } } }
I20260929 10:35:41.985365 29243 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f2e41826e584a2e8864dd419989f112 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 10:35:41.985590 29294 mvcc.cc:205] Tried to move back new op lower bound from 7334617669526728704 to 7334617669505191936. Current Snapshot: MvccSnapshot[applied={T|T < 7334617669526728704}]
I20260929 10:35:41.985713 29261 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85aa0a25b95441f68dec06be351a1674 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b222a401dd004f67b25de657fcd29257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f2e41826e584a2e8864dd419989f112" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 35179 } } peers { permanent_uuid: "b222a401dd004f67b25de657fcd29257" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 44731 } } peers { permanent_uuid: "85aa0a25b95441f68dec06be351a1674" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 34231 } } }
I20260929 10:35:41.985822 29261 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85aa0a25b95441f68dec06be351a1674 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 10:35:42.004634 29292 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:35:42.005785 29292 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:35:42.012785 29292 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b222a401dd004f67b25de657fcd29257: Generated new TSK 0
I20260929 10:35:42.013682 29292 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 10:35:42.723475 29254 catalog_manager.cc:1259] Loaded cluster ID: c129e473c5b84b9b88bf5d2c6e4466d5
I20260929 10:35:42.723558 29254 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6f2e41826e584a2e8864dd419989f112: loading cluster ID for follower catalog manager: success
I20260929 10:35:42.725277 29254 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6f2e41826e584a2e8864dd419989f112: acquiring CA information for follower catalog manager: success
I20260929 10:35:42.726660 29254 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6f2e41826e584a2e8864dd419989f112: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 10:35:42.740773 29287 catalog_manager.cc:1259] Loaded cluster ID: c129e473c5b84b9b88bf5d2c6e4466d5
I20260929 10:35:42.740877 29287 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 85aa0a25b95441f68dec06be351a1674: loading cluster ID for follower catalog manager: success
I20260929 10:35:42.742563 29287 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 85aa0a25b95441f68dec06be351a1674: acquiring CA information for follower catalog manager: success
I20260929 10:35:42.743736 29287 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 85aa0a25b95441f68dec06be351a1674: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 10:35:43.083869 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 29015
I20260929 10:35:43.090737 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 29090
I20260929 10:35:43.096721 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 29166
2026-09-29T10:35:43Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3241 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790677981592896-15689-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:35:43 dist-test-slave-vg89 krb5kdc[29308](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:35:43 dist-test-slave-vg89 krb5kdc[29308](info): set up 2 sockets
Sep 29 10:35:43 dist-test-slave-vg89 krb5kdc[29308](info): commencing operation
krb5kdc: starting...
W20260929 10:35:44.134853 15689 mini_kdc.cc:121] Time spent starting KDC: real 1.016s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Sep 29 10:35:44 dist-test-slave-vg89 krb5kdc[29308](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678144, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:35:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:35:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.82.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab.
Entry for principal kudu/127.15.82.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab.
I20260929 10:35:44.262180 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790677981592896-15689-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790677981592896-15689-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790677981592896-15689-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:34501
--webserver_interface=127.15.82.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab
--principal=kudu/127.15.82.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:45523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.82.126:34501
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:35:44.372974 29321 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:35:44.373324 29321 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:35:44.373409 29321 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:35:44.377619 29321 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:35:44.377732 29321 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:35:44.377774 29321 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:35:44.377813 29321 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:35:44.377854 29321 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:35:44.383591 29321 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:45523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790677981592896-15689-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790677981592896-15689-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.82.126:34501
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab
--principal=kudu/127.15.82.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:34501
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.82.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29321
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790677981592896-15689-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:35:44.384768 29321 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:35:44.385872 29321 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:35:44.395032 29326 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:35:44.395054 29327 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:35:44.395126 29321 server_base.cc:1034] running on GCE node
W20260929 10:35:44.395226 29329 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:35:44.395861 29321 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:35:44.397101 29321 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:35:44.398276 29321 hybrid_clock.cc:648] HybridClock initialized: now 1790678144398240 us; error 50 us; skew 500 ppm
Sep 29 10:35:44 dist-test-slave-vg89 krb5kdc[29308](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678144, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:35:44.401335 29321 init.cc:377] Logged in from keytab as kudu/127.15.82.126@KRBTEST.COM (short username kudu)
I20260929 10:35:44.402325 29321 webserver.cc:460] Webserver started at http://127.15.82.126:34505/ using document root <none> and password file <none>
I20260929 10:35:44.402984 29321 fs_manager.cc:359] Metadata directory not provided
I20260929 10:35:44.403080 29321 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:35:44.403329 29321 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:35:44.405489 29321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790677981592896-15689-0/minicluster-data/master-0/data/instance:
uuid: "30dd6f153a9447ba87602f70d8b324b0"
format_stamp: "Formatted at 2026-09-29 10:35:44 on dist-test-slave-vg89"
server_key: "035ffc16731219d98985304cc5c61c12"
server_key_iv: "b2bfdd5607a857091f3b320b3f2149a2"
server_key_version: "encryptionkey@0"
I20260929 10:35:44.406076 29321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790677981592896-15689-0/minicluster-data/master-0/wal/instance:
uuid: "30dd6f153a9447ba87602f70d8b324b0"
format_stamp: "Formatted at 2026-09-29 10:35:44 on dist-test-slave-vg89"
server_key: "035ffc16731219d98985304cc5c61c12"
server_key_iv: "b2bfdd5607a857091f3b320b3f2149a2"
server_key_version: "encryptionkey@0"
I20260929 10:35:44.412212 29321 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260929 10:35:44.414898 29336 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:35:44.416152 29321 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 10:35:44.416312 29321 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790677981592896-15689-0/minicluster-data/master-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790677981592896-15689-0/minicluster-data/master-0/wal
uuid: "30dd6f153a9447ba87602f70d8b324b0"
format_stamp: "Formatted at 2026-09-29 10:35:44 on dist-test-slave-vg89"
server_key: "035ffc16731219d98985304cc5c61c12"
server_key_iv: "b2bfdd5607a857091f3b320b3f2149a2"
server_key_version: "encryptionkey@0"
I20260929 10:35:44.416435 29321 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790677981592896-15689-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790677981592896-15689-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790677981592896-15689-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:35:44.429077 29321 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:35:44.432482 29321 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:35:44.432693 29321 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:35:44.439543 29321 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.126:34501
I20260929 10:35:44.439566 29388 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.126:34501 every 8 connection(s)
I20260929 10:35:44.440469 29321 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
I20260929 10:35:44.443140 29389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:35:44.448578 29389 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30dd6f153a9447ba87602f70d8b324b0: Bootstrap starting.
I20260929 10:35:44.448935 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 29321
I20260929 10:35:44.449038 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790677981592896-15689-0/minicluster-data/master-0/wal/instance
I20260929 10:35:44.449333 15689 external_mini_cluster.cc:1428] Setting key 2975d63c593833f3a3af1a66efec3638
I20260929 10:35:44.451179 29389 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 30dd6f153a9447ba87602f70d8b324b0: Neither blocks nor log segments found. Creating new log.
I20260929 10:35:44.451886 29389 log.cc:824] T 00000000000000000000000000000000 P 30dd6f153a9447ba87602f70d8b324b0: Log is configured to *not* fsync() on all Append() calls
I20260929 10:35:44.453717 29389 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30dd6f153a9447ba87602f70d8b324b0: No bootstrap required, opened a new log
I20260929 10:35:44.457036 29389 raft_consensus.cc:348] T 00000000000000000000000000000000 P 30dd6f153a9447ba87602f70d8b324b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30dd6f153a9447ba87602f70d8b324b0" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 34501 } }
I20260929 10:35:44.457267 29389 raft_consensus.cc:374] T 00000000000000000000000000000000 P 30dd6f153a9447ba87602f70d8b324b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:35:44.457335 29389 raft_consensus.cc:729] T 00000000000000000000000000000000 P 30dd6f153a9447ba87602f70d8b324b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30dd6f153a9447ba87602f70d8b324b0, State: Initialized, Role: FOLLOWER
Sep 29 10:35:44 dist-test-slave-vg89 krb5kdc[29308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678144, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:35:44.457818 29389 consensus_queue.cc:260] T 00000000000000000000000000000000 P 30dd6f153a9447ba87602f70d8b324b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30dd6f153a9447ba87602f70d8b324b0" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 34501 } }
I20260929 10:35:44.457979 29389 raft_consensus.cc:388] T 00000000000000000000000000000000 P 30dd6f153a9447ba87602f70d8b324b0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:35:44.458060 29389 raft_consensus.cc:482] T 00000000000000000000000000000000 P 30dd6f153a9447ba87602f70d8b324b0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:35:44.458156 29389 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 30dd6f153a9447ba87602f70d8b324b0 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:35:44.459065 29389 raft_consensus.cc:504] T 00000000000000000000000000000000 P 30dd6f153a9447ba87602f70d8b324b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30dd6f153a9447ba87602f70d8b324b0" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 34501 } }
I20260929 10:35:44.459394 29389 leader_election.cc:302] T 00000000000000000000000000000000 P 30dd6f153a9447ba87602f70d8b324b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 30dd6f153a9447ba87602f70d8b324b0; no voters:
I20260929 10:35:44.459687 29389 leader_election.cc:288] T 00000000000000000000000000000000 P 30dd6f153a9447ba87602f70d8b324b0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:35:44.459794 29394 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 30dd6f153a9447ba87602f70d8b324b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:35:44.460036 29394 raft_consensus.cc:686] T 00000000000000000000000000000000 P 30dd6f153a9447ba87602f70d8b324b0 [term 1 LEADER]: Becoming Leader. State: Replica: 30dd6f153a9447ba87602f70d8b324b0, State: Running, Role: LEADER
I20260929 10:35:44.460397 29394 consensus_queue.cc:237] T 00000000000000000000000000000000 P 30dd6f153a9447ba87602f70d8b324b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30dd6f153a9447ba87602f70d8b324b0" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 34501 } }
I20260929 10:35:44.460986 29389 sys_catalog.cc:564] T 00000000000000000000000000000000 P 30dd6f153a9447ba87602f70d8b324b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:35:44.462364 29395 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30dd6f153a9447ba87602f70d8b324b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "30dd6f153a9447ba87602f70d8b324b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30dd6f153a9447ba87602f70d8b324b0" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 34501 } } }
I20260929 10:35:44.462371 29396 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30dd6f153a9447ba87602f70d8b324b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 30dd6f153a9447ba87602f70d8b324b0. Latest consensus state: current_term: 1 leader_uuid: "30dd6f153a9447ba87602f70d8b324b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30dd6f153a9447ba87602f70d8b324b0" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 34501 } } }
I20260929 10:35:44.462605 29395 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30dd6f153a9447ba87602f70d8b324b0 [sys.catalog]: This master's current role is: LEADER
I20260929 10:35:44.462610 29396 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30dd6f153a9447ba87602f70d8b324b0 [sys.catalog]: This master's current role is: LEADER
I20260929 10:35:44.463064 29405 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:35:44.464457 29392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:44.450630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51406 (local address 127.15.82.126:34501)
0929 10:35:44.451042 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0929 10:35:44.451057 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 10:35:44.451090 (+ 33us) server_negotiation.cc:407] Connection header received
0929 10:35:44.451752 (+ 662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:35:44.451771 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:35:44.452140 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:35:44.452475 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:35:44.453281 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:44.454310 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:44.455168 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:44.455640 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:44.455812 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:44.458486 (+ 2674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:35:44.458491 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:35:44.458504 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:35:44.458528 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:35:44.460630 (+ 2102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:35:44.461184 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:35:44.461190 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:35:44.461196 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:35:44.461341 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:35:44.461650 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:35:44.461653 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:35:44.461655 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:35:44.462185 (+ 530us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:35:44.462377 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0929 10:35:44.463255 (+ 878us) server_negotiation.cc:299] Negotiation successful
0929 10:35:44.463562 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":118,"threads_started":1}
I20260929 10:35:44.466238 29405 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:35:44.467173 15689 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 10:35:44.471752 29405 catalog_manager.cc:1347] Generated new cluster ID: fe00e453940c4448ba76ac14f4325831
I20260929 10:35:44.471814 29405 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:35:44.493083 29405 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:35:44.494077 29405 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:35:44.499707 29405 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 30dd6f153a9447ba87602f70d8b324b0: Generated new TSK 0
I20260929 10:35:44.500411 29405 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 10:35:44.523360 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 29321
2026-09-29T10:35:44Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1424 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:35:44 dist-test-slave-vg89 krb5kdc[29416](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:35:44 dist-test-slave-vg89 krb5kdc[29416](info): set up 2 sockets
Sep 29 10:35:44 dist-test-slave-vg89 krb5kdc[29416](info): commencing operation
krb5kdc: starting...
W20260929 10:35:45.583086 15689 mini_kdc.cc:121] Time spent starting KDC: real 1.039s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Sep 29 10:35:45 dist-test-slave-vg89 krb5kdc[29416](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678145, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:35:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:35:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.82.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab.
Entry for principal kudu/127.15.82.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab.
I20260929 10:35:45.713608 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:37881
--webserver_interface=127.15.82.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab
--principal=kudu/127.15.82.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:44075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.82.126:37881
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:35:45.824800 29429 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:35:45.825057 29429 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:35:45.825096 29429 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:35:45.829011 29429 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:35:45.829070 29429 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:35:45.829090 29429 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:35:45.829106 29429 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:35:45.829121 29429 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:35:45.834242 29429 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:44075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.82.126:37881
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab
--principal=kudu/127.15.82.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:37881
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.82.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29429
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:35:45.835259 29429 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:35:45.836194 29429 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:35:45.844771 29434 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:35:45.844754 29437 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:35:45.844731 29435 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:35:45.845065 29429 server_base.cc:1034] running on GCE node
I20260929 10:35:45.845644 29429 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:35:45.846782 29429 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:35:45.847965 29429 hybrid_clock.cc:648] HybridClock initialized: now 1790678145847943 us; error 42 us; skew 500 ppm
Sep 29 10:35:45 dist-test-slave-vg89 krb5kdc[29416](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678145, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:35:45.856673 29429 init.cc:377] Logged in from keytab as kudu/127.15.82.126@KRBTEST.COM (short username kudu)
I20260929 10:35:45.857653 29429 webserver.cc:460] Webserver started at http://127.15.82.126:45403/ using document root <none> and password file <none>
I20260929 10:35:45.858248 29429 fs_manager.cc:359] Metadata directory not provided
I20260929 10:35:45.858340 29429 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:35:45.858603 29429 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:35:45.860689 29429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/master-0/data/instance:
uuid: "00ca51f69920468e9052d7591f5043ec"
format_stamp: "Formatted at 2026-09-29 10:35:45 on dist-test-slave-vg89"
server_key: "70936bd37ac113f4769fa6afe351f761"
server_key_iv: "a921de9c6f3d8ac2d245aab071461444"
server_key_version: "encryptionkey@0"
I20260929 10:35:45.861251 29429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/master-0/wal/instance:
uuid: "00ca51f69920468e9052d7591f5043ec"
format_stamp: "Formatted at 2026-09-29 10:35:45 on dist-test-slave-vg89"
server_key: "70936bd37ac113f4769fa6afe351f761"
server_key_iv: "a921de9c6f3d8ac2d245aab071461444"
server_key_version: "encryptionkey@0"
I20260929 10:35:45.865043 29429 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260929 10:35:45.867559 29444 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:35:45.868753 29429 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 10:35:45.868906 29429 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/master-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/master-0/wal
uuid: "00ca51f69920468e9052d7591f5043ec"
format_stamp: "Formatted at 2026-09-29 10:35:45 on dist-test-slave-vg89"
server_key: "70936bd37ac113f4769fa6afe351f761"
server_key_iv: "a921de9c6f3d8ac2d245aab071461444"
server_key_version: "encryptionkey@0"
I20260929 10:35:45.869028 29429 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:35:45.893569 29429 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:35:45.896596 29429 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:35:45.896806 29429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:35:45.904168 29429 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.126:37881
I20260929 10:35:45.904469 29496 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.126:37881 every 8 connection(s)
I20260929 10:35:45.905174 29429 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
I20260929 10:35:45.907943 29497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:35:45.911072 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 29429
I20260929 10:35:45.911170 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/master-0/wal/instance
I20260929 10:35:45.911473 15689 external_mini_cluster.cc:1428] Setting key 5ab941f950eb39de5cb58c85c97bdd4b
I20260929 10:35:45.913748 29497 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00ca51f69920468e9052d7591f5043ec: Bootstrap starting.
I20260929 10:35:45.916323 29497 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 00ca51f69920468e9052d7591f5043ec: Neither blocks nor log segments found. Creating new log.
I20260929 10:35:45.917165 29497 log.cc:824] T 00000000000000000000000000000000 P 00ca51f69920468e9052d7591f5043ec: Log is configured to *not* fsync() on all Append() calls
I20260929 10:35:45.919119 29497 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00ca51f69920468e9052d7591f5043ec: No bootstrap required, opened a new log
Sep 29 10:35:45 dist-test-slave-vg89 krb5kdc[29416](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678145, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:35:45.922597 29497 raft_consensus.cc:348] T 00000000000000000000000000000000 P 00ca51f69920468e9052d7591f5043ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ca51f69920468e9052d7591f5043ec" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 37881 } }
I20260929 10:35:45.922840 29497 raft_consensus.cc:374] T 00000000000000000000000000000000 P 00ca51f69920468e9052d7591f5043ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:35:45.922935 29497 raft_consensus.cc:729] T 00000000000000000000000000000000 P 00ca51f69920468e9052d7591f5043ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00ca51f69920468e9052d7591f5043ec, State: Initialized, Role: FOLLOWER
I20260929 10:35:45.923466 29497 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00ca51f69920468e9052d7591f5043ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ca51f69920468e9052d7591f5043ec" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 37881 } }
I20260929 10:35:45.923645 29497 raft_consensus.cc:388] T 00000000000000000000000000000000 P 00ca51f69920468e9052d7591f5043ec [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:35:45.923724 29497 raft_consensus.cc:482] T 00000000000000000000000000000000 P 00ca51f69920468e9052d7591f5043ec [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:35:45.923848 29497 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 00ca51f69920468e9052d7591f5043ec [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:35:45.924965 29497 raft_consensus.cc:504] T 00000000000000000000000000000000 P 00ca51f69920468e9052d7591f5043ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ca51f69920468e9052d7591f5043ec" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 37881 } }
I20260929 10:35:45.925319 29497 leader_election.cc:302] T 00000000000000000000000000000000 P 00ca51f69920468e9052d7591f5043ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 00ca51f69920468e9052d7591f5043ec; no voters:
I20260929 10:35:45.925633 29497 leader_election.cc:288] T 00000000000000000000000000000000 P 00ca51f69920468e9052d7591f5043ec [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:35:45.925734 29502 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 00ca51f69920468e9052d7591f5043ec [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:35:45.925935 29502 raft_consensus.cc:686] T 00000000000000000000000000000000 P 00ca51f69920468e9052d7591f5043ec [term 1 LEADER]: Becoming Leader. State: Replica: 00ca51f69920468e9052d7591f5043ec, State: Running, Role: LEADER
I20260929 10:35:45.926035 29500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:45.912979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46988 (local address 127.15.82.126:37881)
0929 10:35:45.913385 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0929 10:35:45.913401 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0929 10:35:45.913437 (+ 36us) server_negotiation.cc:407] Connection header received
0929 10:35:45.914100 (+ 663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:35:45.914119 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:35:45.914508 (+ 389us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:35:45.914816 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:35:45.915743 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:45.916782 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:45.917881 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:45.918352 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:45.918604 (+ 252us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:45.921044 (+ 2440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:35:45.921049 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:35:45.921062 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:35:45.921088 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:35:45.922889 (+ 1801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:35:45.923377 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:35:45.923382 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:35:45.923386 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:35:45.923505 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:35:45.923829 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:35:45.923832 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:35:45.923834 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:35:45.924304 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:35:45.924455 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0929 10:35:45.924748 (+ 293us) server_negotiation.cc:299] Negotiation successful
0929 10:35:45.925029 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":129,"threads_started":1}
I20260929 10:35:45.926304 29502 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00ca51f69920468e9052d7591f5043ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ca51f69920468e9052d7591f5043ec" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 37881 } }
I20260929 10:35:45.926988 29497 sys_catalog.cc:564] T 00000000000000000000000000000000 P 00ca51f69920468e9052d7591f5043ec [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:35:45.928162 29504 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00ca51f69920468e9052d7591f5043ec [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00ca51f69920468e9052d7591f5043ec. Latest consensus state: current_term: 1 leader_uuid: "00ca51f69920468e9052d7591f5043ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ca51f69920468e9052d7591f5043ec" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 37881 } } }
I20260929 10:35:45.928169 29503 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00ca51f69920468e9052d7591f5043ec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "00ca51f69920468e9052d7591f5043ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ca51f69920468e9052d7591f5043ec" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 37881 } } }
I20260929 10:35:45.928335 29504 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00ca51f69920468e9052d7591f5043ec [sys.catalog]: This master's current role is: LEADER
I20260929 10:35:45.928354 29503 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00ca51f69920468e9052d7591f5043ec [sys.catalog]: This master's current role is: LEADER
W20260929 10:35:45.931645 29517 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 00ca51f69920468e9052d7591f5043ec: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 10:35:45.931720 29517 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 10:35:45.931794 29513 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:35:45.932483 29513 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:35:45.938221 29513 catalog_manager.cc:1347] Generated new cluster ID: d070de294db6467bb3d6ee46974440f3
I20260929 10:35:45.938284 29513 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:35:45.969530 29513 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:35:45.970919 29513 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:35:45.984331 29513 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 00ca51f69920468e9052d7591f5043ec: Generated new TSK 0
I20260929 10:35:45.985229 29513 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.82.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab.
Entry for principal kudu/127.15.82.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab.
I20260929 10:35:46.016999 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.65:0
--local_ip_for_outbound_sockets=127.15.82.65
--webserver_interface=127.15.82.65
--webserver_port=0
--tserver_master_addrs=127.15.82.126:37881
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab
--principal=kudu/127.15.82.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:44075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:35:46.132067 29523 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:35:46.132373 29523 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:35:46.132450 29523 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:35:46.136528 29523 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:35:46.136618 29523 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:35:46.136714 29523 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.65
I20260929 10:35:46.142004 29523 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:44075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab
--principal=kudu/127.15.82.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.82.65
--webserver_port=0
--tserver_master_addrs=127.15.82.126:37881
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29523
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.65
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:35:46.143340 29523 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:35:46.144474 29523 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:35:46.153993 29531 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:35:46.154047 29528 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:35:46.154083 29529 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:35:46.154479 29523 server_base.cc:1034] running on GCE node
I20260929 10:35:46.154892 29523 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:35:46.155521 29523 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:35:46.156706 29523 hybrid_clock.cc:648] HybridClock initialized: now 1790678146156677 us; error 52 us; skew 500 ppm
Sep 29 10:35:46 dist-test-slave-vg89 krb5kdc[29416](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678146, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:35:46.159545 29523 init.cc:377] Logged in from keytab as kudu/127.15.82.65@KRBTEST.COM (short username kudu)
I20260929 10:35:46.160386 29523 webserver.cc:460] Webserver started at http://127.15.82.65:35431/ using document root <none> and password file <none>
I20260929 10:35:46.160959 29523 fs_manager.cc:359] Metadata directory not provided
I20260929 10:35:46.161019 29523 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:35:46.161257 29523 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:35:46.163349 29523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-0/data/instance:
uuid: "f379905b6ee84cc6bacf15584dcfd086"
format_stamp: "Formatted at 2026-09-29 10:35:46 on dist-test-slave-vg89"
server_key: "412b7efc5455c0d3d9eb9b84117cdc25"
server_key_iv: "3d151e1d30376524fe048ce203a8084e"
server_key_version: "encryptionkey@0"
I20260929 10:35:46.163923 29523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-0/wal/instance:
uuid: "f379905b6ee84cc6bacf15584dcfd086"
format_stamp: "Formatted at 2026-09-29 10:35:46 on dist-test-slave-vg89"
server_key: "412b7efc5455c0d3d9eb9b84117cdc25"
server_key_iv: "3d151e1d30376524fe048ce203a8084e"
server_key_version: "encryptionkey@0"
I20260929 10:35:46.167676 29523 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260929 10:35:46.170082 29538 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:35:46.171358 29523 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 10:35:46.171489 29523 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-0/wal
uuid: "f379905b6ee84cc6bacf15584dcfd086"
format_stamp: "Formatted at 2026-09-29 10:35:46 on dist-test-slave-vg89"
server_key: "412b7efc5455c0d3d9eb9b84117cdc25"
server_key_iv: "3d151e1d30376524fe048ce203a8084e"
server_key_version: "encryptionkey@0"
I20260929 10:35:46.171589 29523 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:35:46.206565 29523 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:35:46.210038 29523 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:35:46.210259 29523 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:35:46.210927 29523 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:35:46.211766 29523 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:35:46.211813 29523 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:35:46.211879 29523 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:35:46.211910 29523 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:35:46.221285 29523 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.65:34003
I20260929 10:35:46.221304 29651 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.65:34003 every 8 connection(s)
I20260929 10:35:46.222208 29523 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
I20260929 10:35:46.224406 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 29523
I20260929 10:35:46.224545 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-0/wal/instance
I20260929 10:35:46.224846 15689 external_mini_cluster.cc:1428] Setting key 6b0154d67e7feaf9f3c1b1ae3b56f60f
Sep 29 10:35:46 dist-test-slave-vg89 krb5kdc[29416](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678146, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.65@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.82.66@KRBTEST.COM; defaulting to no policy
I20260929 10:35:46.236914 29500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:46.224085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.65:51571 (local address 127.15.82.126:37881)
0929 10:35:46.224270 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0929 10:35:46.224278 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 10:35:46.225211 (+ 933us) server_negotiation.cc:407] Connection header received
0929 10:35:46.226762 (+ 1551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:35:46.226767 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:35:46.226826 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:35:46.226920 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:35:46.228336 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:46.228990 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:46.230150 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:46.230456 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:46.230573 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:46.233007 (+ 2434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:35:46.233010 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:35:46.233013 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:35:46.233046 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:35:46.234692 (+ 1646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:35:46.235243 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:35:46.235248 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:35:46.235250 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:35:46.235350 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:35:46.235715 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:35:46.235720 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:35:46.235722 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:35:46.235913 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:35:46.236022 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0929 10:35:46.236597 (+ 575us) server_negotiation.cc:299] Negotiation successful
0929 10:35:46.236731 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.66@KRBTEST.COM" created.
I20260929 10:35:46.237800 29654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:46.224488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:37881 (local address 127.15.82.65:51571)
0929 10:35:46.225033 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:46.225088 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0929 10:35:46.226564 (+ 1476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:35:46.227163 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:35:46.227172 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:35:46.227630 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:35:46.228185 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:46.228200 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:46.229121 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:46.229125 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:46.230015 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:46.230024 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:46.230628 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:46.230632 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:46.230792 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:46.231407 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:35:46.231425 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:35:46.232844 (+ 1419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:35:46.234818 (+ 1974us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:35:46.234823 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:35:46.234840 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:35:46.235111 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:35:46.235465 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:35:46.235469 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:35:46.235471 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:35:46.235619 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:35:46.236021 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:35:46.236030 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:35:46.236348 (+ 318us) client_negotiation.cc:769] Sending connection context
0929 10:35:46.236542 (+ 194us) client_negotiation.cc:240] Negotiation successful
0929 10:35:46.236961 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":136,"threads_started":1}
I20260929 10:35:46.239251 29652 heartbeater.cc:347] Connected to a master server at 127.15.82.126:37881
I20260929 10:35:46.239547 29652 heartbeater.cc:464] Registering TS with master...
I20260929 10:35:46.240217 29652 heartbeater.cc:511] Master 127.15.82.126:37881 requested a full tablet report, sending...
I20260929 10:35:46.241863 29461 ts_manager.cc:194] Registered new tserver with Master: f379905b6ee84cc6bacf15584dcfd086 (127.15.82.65:34003)
I20260929 10:35:46.243085 29461 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.82.65@KRBTEST.COM'} at 127.15.82.65:51571
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab.
Entry for principal kudu/127.15.82.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab.
I20260929 10:35:46.254338 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.66:0
--local_ip_for_outbound_sockets=127.15.82.66
--webserver_interface=127.15.82.66
--webserver_port=0
--tserver_master_addrs=127.15.82.126:37881
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab
--principal=kudu/127.15.82.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:44075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:35:46.365772 29657 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:35:46.366050 29657 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:35:46.366096 29657 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:35:46.370075 29657 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:35:46.370141 29657 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:35:46.370203 29657 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.66
I20260929 10:35:46.375780 29657 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:44075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab
--principal=kudu/127.15.82.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.82.66
--webserver_port=0
--tserver_master_addrs=127.15.82.126:37881
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29657
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.66
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:35:46.376840 29657 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:35:46.377830 29657 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:35:46.387367 29665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:35:46.387408 29657 server_base.cc:1034] running on GCE node
W20260929 10:35:46.387369 29662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:35:46.387369 29663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:35:46.388048 29657 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:35:46.388693 29657 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:35:46.389897 29657 hybrid_clock.cc:648] HybridClock initialized: now 1790678146389850 us; error 52 us; skew 500 ppm
Sep 29 10:35:46 dist-test-slave-vg89 krb5kdc[29416](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678146, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:35:46.392567 29657 init.cc:377] Logged in from keytab as kudu/127.15.82.66@KRBTEST.COM (short username kudu)
I20260929 10:35:46.393520 29657 webserver.cc:460] Webserver started at http://127.15.82.66:40525/ using document root <none> and password file <none>
I20260929 10:35:46.394080 29657 fs_manager.cc:359] Metadata directory not provided
I20260929 10:35:46.394166 29657 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:35:46.394438 29657 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:35:46.396472 29657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-1/data/instance:
uuid: "c53d47f2877d47dc9e0e6d6793562c71"
format_stamp: "Formatted at 2026-09-29 10:35:46 on dist-test-slave-vg89"
server_key: "fa1f3b5e7b477f15c03b5a1d21684acc"
server_key_iv: "0ca6a0f33ea9194c85e94ea0e0e92a63"
server_key_version: "encryptionkey@0"
I20260929 10:35:46.397045 29657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-1/wal/instance:
uuid: "c53d47f2877d47dc9e0e6d6793562c71"
format_stamp: "Formatted at 2026-09-29 10:35:46 on dist-test-slave-vg89"
server_key: "fa1f3b5e7b477f15c03b5a1d21684acc"
server_key_iv: "0ca6a0f33ea9194c85e94ea0e0e92a63"
server_key_version: "encryptionkey@0"
I20260929 10:35:46.401299 29657 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260929 10:35:46.403873 29672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:35:46.405092 29657 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260929 10:35:46.405241 29657 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-1/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-1/wal
uuid: "c53d47f2877d47dc9e0e6d6793562c71"
format_stamp: "Formatted at 2026-09-29 10:35:46 on dist-test-slave-vg89"
server_key: "fa1f3b5e7b477f15c03b5a1d21684acc"
server_key_iv: "0ca6a0f33ea9194c85e94ea0e0e92a63"
server_key_version: "encryptionkey@0"
I20260929 10:35:46.405354 29657 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:35:46.426666 29657 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:35:46.430449 29657 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:35:46.430685 29657 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:35:46.431337 29657 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:35:46.432247 29657 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:35:46.432299 29657 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:35:46.432401 29657 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:35:46.432444 29657 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:35:46.441720 29657 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.66:38923
I20260929 10:35:46.441738 29785 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.66:38923 every 8 connection(s)
I20260929 10:35:46.442781 29657 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
I20260929 10:35:46.451372 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 29657
I20260929 10:35:46.451511 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-1/wal/instance
I20260929 10:35:46.451802 15689 external_mini_cluster.cc:1428] Setting key d0351174516d553fea1170370b4260e6
Sep 29 10:35:46 dist-test-slave-vg89 krb5kdc[29416](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678146, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.66@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:35:46.457141 29500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:46.444685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:40147 (local address 127.15.82.126:37881)
0929 10:35:46.444828 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0929 10:35:46.444832 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:35:46.445751 (+ 919us) server_negotiation.cc:407] Connection header received
0929 10:35:46.447225 (+ 1474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:35:46.447228 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:35:46.447289 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:35:46.447393 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:35:46.448597 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:46.449143 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:46.450317 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:46.450610 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:46.450721 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:46.453550 (+ 2829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:35:46.453554 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:35:46.453556 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:35:46.453580 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:35:46.455069 (+ 1489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:35:46.455596 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:35:46.455601 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:35:46.455603 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:35:46.455684 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:35:46.456041 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:35:46.456046 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:35:46.456048 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:35:46.456233 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:35:46.456341 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0929 10:35:46.456841 (+ 500us) server_negotiation.cc:299] Negotiation successful
0929 10:35:46.456972 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 10:35:46.458098 29788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:46.445106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:37881 (local address 127.15.82.66:40147)
0929 10:35:46.445596 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:46.445641 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0929 10:35:46.446989 (+ 1348us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:35:46.447582 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:35:46.447590 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:35:46.447951 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:35:46.448439 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:46.448453 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:46.449272 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:46.449280 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:46.450188 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:46.450198 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:46.450763 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:46.450767 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:46.450951 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:46.451676 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:35:46.451692 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:35:46.453381 (+ 1689us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:35:46.455191 (+ 1810us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:35:46.455197 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:35:46.455213 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:35:46.455493 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:35:46.455785 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:35:46.455788 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:35:46.455790 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:35:46.455945 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:35:46.456339 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:35:46.456346 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:35:46.456603 (+ 257us) client_negotiation.cc:769] Sending connection context
0929 10:35:46.456802 (+ 199us) client_negotiation.cc:240] Negotiation successful
0929 10:35:46.457157 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":119,"threads_started":1}
I20260929 10:35:46.459458 29786 heartbeater.cc:347] Connected to a master server at 127.15.82.126:37881
I20260929 10:35:46.459717 29786 heartbeater.cc:464] Registering TS with master...
I20260929 10:35:46.460299 29786 heartbeater.cc:511] Master 127.15.82.126:37881 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.15.82.67@KRBTEST.COM; defaulting to no policy
I20260929 10:35:46.461488 29461 ts_manager.cc:194] Registered new tserver with Master: c53d47f2877d47dc9e0e6d6793562c71 (127.15.82.66:38923)
I20260929 10:35:46.462302 29461 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.82.66@KRBTEST.COM'} at 127.15.82.66:40147
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab.
Entry for principal kudu/127.15.82.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab.
I20260929 10:35:46.481887 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.67:0
--local_ip_for_outbound_sockets=127.15.82.67
--webserver_interface=127.15.82.67
--webserver_port=0
--tserver_master_addrs=127.15.82.126:37881
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab
--principal=kudu/127.15.82.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:44075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:35:46.595927 29791 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:35:46.596185 29791 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:35:46.596261 29791 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:35:46.600256 29791 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:35:46.600310 29791 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:35:46.600406 29791 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.67
I20260929 10:35:46.605566 29791 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:44075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab
--principal=kudu/127.15.82.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.82.67
--webserver_port=0
--tserver_master_addrs=127.15.82.126:37881
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29791
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.67
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:35:46.606674 29791 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:35:46.607697 29791 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:35:46.617395 29796 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:35:46.617403 29797 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:35:46.617453 29791 server_base.cc:1034] running on GCE node
W20260929 10:35:46.617410 29799 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:35:46.618041 29791 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:35:46.618742 29791 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:35:46.619962 29791 hybrid_clock.cc:648] HybridClock initialized: now 1790678146619945 us; error 28 us; skew 500 ppm
Sep 29 10:35:46 dist-test-slave-vg89 krb5kdc[29416](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678146, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:35:46.622699 29791 init.cc:377] Logged in from keytab as kudu/127.15.82.67@KRBTEST.COM (short username kudu)
I20260929 10:35:46.623564 29791 webserver.cc:460] Webserver started at http://127.15.82.67:37299/ using document root <none> and password file <none>
I20260929 10:35:46.624126 29791 fs_manager.cc:359] Metadata directory not provided
I20260929 10:35:46.624217 29791 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:35:46.624447 29791 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:35:46.626523 29791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-2/data/instance:
uuid: "ee6ba208af054a77a67f4e0598cd8c63"
format_stamp: "Formatted at 2026-09-29 10:35:46 on dist-test-slave-vg89"
server_key: "3ecac6d030922b70a081b8bd106edd1c"
server_key_iv: "49980e6545024fe31ced90f03bc88c82"
server_key_version: "encryptionkey@0"
I20260929 10:35:46.627091 29791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-2/wal/instance:
uuid: "ee6ba208af054a77a67f4e0598cd8c63"
format_stamp: "Formatted at 2026-09-29 10:35:46 on dist-test-slave-vg89"
server_key: "3ecac6d030922b70a081b8bd106edd1c"
server_key_iv: "49980e6545024fe31ced90f03bc88c82"
server_key_version: "encryptionkey@0"
I20260929 10:35:46.631002 29791 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260929 10:35:46.633610 29806 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:35:46.634974 29791 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 10:35:46.635123 29791 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-2/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-2/wal
uuid: "ee6ba208af054a77a67f4e0598cd8c63"
format_stamp: "Formatted at 2026-09-29 10:35:46 on dist-test-slave-vg89"
server_key: "3ecac6d030922b70a081b8bd106edd1c"
server_key_iv: "49980e6545024fe31ced90f03bc88c82"
server_key_version: "encryptionkey@0"
I20260929 10:35:46.635245 29791 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:35:46.655447 29791 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:35:46.659150 29791 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:35:46.659382 29791 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:35:46.660025 29791 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:35:46.660941 29791 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:35:46.661031 29791 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:35:46.661108 29791 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:35:46.661164 29791 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:35:46.671217 29791 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.67:38555
I20260929 10:35:46.671238 29919 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.67:38555 every 8 connection(s)
I20260929 10:35:46.672195 29791 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
I20260929 10:35:46.679421 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 29791
I20260929 10:35:46.679555 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-2/wal/instance
I20260929 10:35:46.679849 15689 external_mini_cluster.cc:1428] Setting key 14e0ecfa1ab8015a8aab92973a44f736
Sep 29 10:35:46 dist-test-slave-vg89 krb5kdc[29416](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678146, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.67@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:35:46.687299 29500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:46.674216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.67:43047 (local address 127.15.82.126:37881)
0929 10:35:46.674370 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0929 10:35:46.674375 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:35:46.675403 (+ 1028us) server_negotiation.cc:407] Connection header received
0929 10:35:46.676906 (+ 1503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:35:46.676908 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:35:46.676958 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:35:46.677062 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:35:46.678347 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:46.679019 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:46.680725 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:46.681055 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:46.681172 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:46.683981 (+ 2809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:35:46.683984 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:35:46.683987 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:35:46.684012 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:35:46.685408 (+ 1396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:35:46.685912 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:35:46.685914 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:35:46.685915 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:35:46.685993 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:35:46.686291 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:35:46.686293 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:35:46.686294 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:35:46.686529 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:35:46.686608 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0929 10:35:46.687075 (+ 467us) server_negotiation.cc:299] Negotiation successful
0929 10:35:46.687173 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260929 10:35:46.688171 29922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:46.674695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:37881 (local address 127.15.82.67:43047)
0929 10:35:46.675153 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:46.675196 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 10:35:46.676671 (+ 1475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:35:46.677274 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:35:46.677282 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:35:46.677654 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:35:46.678161 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:46.678174 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:46.679194 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:46.679200 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:46.680596 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:46.680606 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:46.681221 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:46.681224 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:46.681453 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:46.682134 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:35:46.682158 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:35:46.683814 (+ 1656us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:35:46.685524 (+ 1710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:35:46.685529 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:35:46.685542 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:35:46.685824 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:35:46.686069 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:35:46.686071 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:35:46.686072 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:35:46.686214 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:35:46.686617 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:35:46.686622 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:35:46.686857 (+ 235us) client_negotiation.cc:769] Sending connection context
0929 10:35:46.687035 (+ 178us) client_negotiation.cc:240] Negotiation successful
0929 10:35:46.687291 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":102,"threads_started":1}
I20260929 10:35:46.689379 29920 heartbeater.cc:347] Connected to a master server at 127.15.82.126:37881
I20260929 10:35:46.689625 29920 heartbeater.cc:464] Registering TS with master...
I20260929 10:35:46.690104 29920 heartbeater.cc:511] Master 127.15.82.126:37881 requested a full tablet report, sending...
I20260929 10:35:46.691197 29461 ts_manager.cc:194] Registered new tserver with Master: ee6ba208af054a77a67f4e0598cd8c63 (127.15.82.67:38555)
I20260929 10:35:46.691828 29461 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.82.67@KRBTEST.COM'} at 127.15.82.67:43047
I20260929 10:35:46.693962 15689 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 10:35:46.704306 29500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:46.696246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47000 (local address 127.15.82.126:37881)
0929 10:35:46.696382 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0929 10:35:46.696388 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 10:35:46.696589 (+ 201us) server_negotiation.cc:407] Connection header received
0929 10:35:46.696755 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:35:46.696759 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:35:46.696806 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:35:46.696934 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:35:46.697691 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:46.698219 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:46.699167 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:46.699412 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:46.699533 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:46.701045 (+ 1512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:35:46.701049 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:35:46.701052 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:35:46.701080 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:35:46.702456 (+ 1376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:35:46.702922 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:35:46.702926 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:35:46.702928 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:35:46.703023 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:35:46.703323 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:35:46.703328 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:35:46.703330 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:35:46.703571 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:35:46.703726 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0929 10:35:46.703988 (+ 262us) server_negotiation.cc:299] Negotiation successful
0929 10:35:46.704114 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260929 10:35:46.707054 29461 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47000:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 10:35:46.710336 29461 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:35:46.725119 29931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:46.719123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.67:38555 (local address 127.0.0.1:54412)
0929 10:35:46.719864 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:46.719918 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0929 10:35:46.720052 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:35:46.720620 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:35:46.720626 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:35:46.720648 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:35:46.720769 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:46.720774 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:46.722392 (+ 1618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:46.722441 (+ 49us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:46.723578 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:46.723588 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:46.724545 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:46.724549 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:46.724655 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:46.724692 (+ 37us) client_negotiation.cc:769] Sending connection context
0929 10:35:46.724798 (+ 106us) client_negotiation.cc:240] Negotiation successful
0929 10:35:46.724920 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":625,"spinlock_wait_cycles":3840,"thread_start_us":90,"threads_started":1}
I20260929 10:35:46.725117 29933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:46.719346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.66:38923 (local address 127.0.0.1:53770)
0929 10:35:46.720286 (+ 940us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:46.720299 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 10:35:46.720381 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:35:46.720881 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:35:46.720883 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:35:46.720899 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:35:46.720995 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:46.721001 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:46.722103 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:46.722109 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:46.723665 (+ 1556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:46.723676 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:46.724772 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:46.724777 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:46.724861 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:46.724881 (+ 20us) client_negotiation.cc:769] Sending connection context
0929 10:35:46.724942 (+ 61us) client_negotiation.cc:240] Negotiation successful
0929 10:35:46.724997 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":883,"mutex_wait_us":31,"thread_start_us":93,"threads_started":1}
I20260929 10:35:46.725445 29936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:46.719736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53770 (local address 127.15.82.66:38923)
0929 10:35:46.720352 (+ 616us) server_negotiation.cc:206] Beginning negotiation
0929 10:35:46.720367 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 10:35:46.720389 (+ 22us) server_negotiation.cc:407] Connection header received
0929 10:35:46.720557 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:35:46.720564 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:35:46.720743 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:35:46.720901 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:35:46.721143 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:46.721970 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:46.723802 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:46.724573 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:46.724687 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:46.724827 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0929 10:35:46.725182 (+ 355us) server_negotiation.cc:299] Negotiation successful
0929 10:35:46.725299 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":512,"thread_start_us":105,"threads_started":1}
I20260929 10:35:46.725857 29932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:46.719126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46022 (local address 127.15.82.65:34003)
0929 10:35:46.720574 (+ 1448us) server_negotiation.cc:206] Beginning negotiation
0929 10:35:46.720588 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 10:35:46.720609 (+ 21us) server_negotiation.cc:407] Connection header received
0929 10:35:46.720674 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:35:46.720678 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:35:46.720831 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:35:46.720970 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:35:46.721327 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:46.722493 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:46.724063 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:46.724840 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:46.724966 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:46.725096 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0929 10:35:46.725628 (+ 532us) server_negotiation.cc:299] Negotiation successful
0929 10:35:46.725720 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1353,"thread_start_us":84,"threads_started":1}
I20260929 10:35:46.726150 29934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:46.718957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.65:34003 (local address 127.0.0.1:46022)
0929 10:35:46.719864 (+ 907us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:46.719918 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0929 10:35:46.720053 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:35:46.721087 (+ 1034us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:35:46.721093 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:35:46.721114 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:35:46.721202 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:46.721208 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:46.722803 (+ 1595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:46.722807 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:46.723937 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:46.723947 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:46.724973 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:46.724976 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:46.725082 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:46.725106 (+ 24us) client_negotiation.cc:769] Sending connection context
0929 10:35:46.725184 (+ 78us) client_negotiation.cc:240] Negotiation successful
0929 10:35:46.726001 (+ 817us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":731,"mutex_wait_us":31,"spinlock_wait_cycles":6528,"thread_start_us":83,"threads_started":1}
I20260929 10:35:46.726706 29935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:46.719609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54412 (local address 127.15.82.67:38555)
0929 10:35:46.720085 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0929 10:35:46.720099 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 10:35:46.720123 (+ 24us) server_negotiation.cc:407] Connection header received
0929 10:35:46.720201 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:35:46.720206 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:35:46.720427 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:35:46.720939 (+ 512us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:35:46.721093 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:46.722259 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:46.723712 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:46.724415 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:46.724535 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:46.724695 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0929 10:35:46.726440 (+ 1745us) server_negotiation.cc:299] Negotiation successful
0929 10:35:46.726540 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":154,"threads_started":1}
I20260929 10:35:46.728802 29586 tablet_service.cc:1467] Processing CreateTablet for tablet ed5bfeedec294bd181422799ebda90ef (DEFAULT_TABLE table=test-table [id=eea4cb7e40544a5ab5d1a3ebc0ec82ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:35:46.728802 29720 tablet_service.cc:1467] Processing CreateTablet for tablet ed5bfeedec294bd181422799ebda90ef (DEFAULT_TABLE table=test-table [id=eea4cb7e40544a5ab5d1a3ebc0ec82ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:35:46.729970 29586 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed5bfeedec294bd181422799ebda90ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:35:46.729970 29720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed5bfeedec294bd181422799ebda90ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:35:46.737505 29937 tablet_bootstrap.cc:492] T ed5bfeedec294bd181422799ebda90ef P f379905b6ee84cc6bacf15584dcfd086: Bootstrap starting.
I20260929 10:35:46.737318 29854 tablet_service.cc:1467] Processing CreateTablet for tablet ed5bfeedec294bd181422799ebda90ef (DEFAULT_TABLE table=test-table [id=eea4cb7e40544a5ab5d1a3ebc0ec82ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:35:46.738101 29938 tablet_bootstrap.cc:492] T ed5bfeedec294bd181422799ebda90ef P c53d47f2877d47dc9e0e6d6793562c71: Bootstrap starting.
I20260929 10:35:46.738652 29854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed5bfeedec294bd181422799ebda90ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:35:46.740141 29937 tablet_bootstrap.cc:654] T ed5bfeedec294bd181422799ebda90ef P f379905b6ee84cc6bacf15584dcfd086: Neither blocks nor log segments found. Creating new log.
I20260929 10:35:46.740646 29938 tablet_bootstrap.cc:654] T ed5bfeedec294bd181422799ebda90ef P c53d47f2877d47dc9e0e6d6793562c71: Neither blocks nor log segments found. Creating new log.
I20260929 10:35:46.741560 29938 log.cc:824] T ed5bfeedec294bd181422799ebda90ef P c53d47f2877d47dc9e0e6d6793562c71: Log is configured to *not* fsync() on all Append() calls
I20260929 10:35:46.741560 29937 log.cc:824] T ed5bfeedec294bd181422799ebda90ef P f379905b6ee84cc6bacf15584dcfd086: Log is configured to *not* fsync() on all Append() calls
I20260929 10:35:46.743453 29937 tablet_bootstrap.cc:492] T ed5bfeedec294bd181422799ebda90ef P f379905b6ee84cc6bacf15584dcfd086: No bootstrap required, opened a new log
I20260929 10:35:46.743681 29937 ts_tablet_manager.cc:1397] T ed5bfeedec294bd181422799ebda90ef P f379905b6ee84cc6bacf15584dcfd086: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260929 10:35:46.744256 29938 tablet_bootstrap.cc:492] T ed5bfeedec294bd181422799ebda90ef P c53d47f2877d47dc9e0e6d6793562c71: No bootstrap required, opened a new log
I20260929 10:35:46.744441 29938 ts_tablet_manager.cc:1397] T ed5bfeedec294bd181422799ebda90ef P c53d47f2877d47dc9e0e6d6793562c71: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260929 10:35:46.745540 29941 tablet_bootstrap.cc:492] T ed5bfeedec294bd181422799ebda90ef P ee6ba208af054a77a67f4e0598cd8c63: Bootstrap starting.
I20260929 10:35:46.747922 29938 raft_consensus.cc:348] T ed5bfeedec294bd181422799ebda90ef P c53d47f2877d47dc9e0e6d6793562c71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f379905b6ee84cc6bacf15584dcfd086" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34003 } } peers { permanent_uuid: "ee6ba208af054a77a67f4e0598cd8c63" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 38555 } } peers { permanent_uuid: "c53d47f2877d47dc9e0e6d6793562c71" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 38923 } }
I20260929 10:35:46.747927 29937 raft_consensus.cc:348] T ed5bfeedec294bd181422799ebda90ef P f379905b6ee84cc6bacf15584dcfd086 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f379905b6ee84cc6bacf15584dcfd086" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34003 } } peers { permanent_uuid: "ee6ba208af054a77a67f4e0598cd8c63" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 38555 } } peers { permanent_uuid: "c53d47f2877d47dc9e0e6d6793562c71" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 38923 } }
I20260929 10:35:46.748201 29938 raft_consensus.cc:374] T ed5bfeedec294bd181422799ebda90ef P c53d47f2877d47dc9e0e6d6793562c71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:35:46.748239 29941 tablet_bootstrap.cc:654] T ed5bfeedec294bd181422799ebda90ef P ee6ba208af054a77a67f4e0598cd8c63: Neither blocks nor log segments found. Creating new log.
I20260929 10:35:46.748274 29937 raft_consensus.cc:374] T ed5bfeedec294bd181422799ebda90ef P f379905b6ee84cc6bacf15584dcfd086 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:35:46.748291 29938 raft_consensus.cc:729] T ed5bfeedec294bd181422799ebda90ef P c53d47f2877d47dc9e0e6d6793562c71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c53d47f2877d47dc9e0e6d6793562c71, State: Initialized, Role: FOLLOWER
I20260929 10:35:46.748327 29937 raft_consensus.cc:729] T ed5bfeedec294bd181422799ebda90ef P f379905b6ee84cc6bacf15584dcfd086 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f379905b6ee84cc6bacf15584dcfd086, State: Initialized, Role: FOLLOWER
I20260929 10:35:46.748741 29938 consensus_queue.cc:260] T ed5bfeedec294bd181422799ebda90ef P c53d47f2877d47dc9e0e6d6793562c71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f379905b6ee84cc6bacf15584dcfd086" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34003 } } peers { permanent_uuid: "ee6ba208af054a77a67f4e0598cd8c63" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 38555 } } peers { permanent_uuid: "c53d47f2877d47dc9e0e6d6793562c71" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 38923 } }
I20260929 10:35:46.748814 29937 consensus_queue.cc:260] T ed5bfeedec294bd181422799ebda90ef P f379905b6ee84cc6bacf15584dcfd086 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f379905b6ee84cc6bacf15584dcfd086" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34003 } } peers { permanent_uuid: "ee6ba208af054a77a67f4e0598cd8c63" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 38555 } } peers { permanent_uuid: "c53d47f2877d47dc9e0e6d6793562c71" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 38923 } }
I20260929 10:35:46.749155 29941 log.cc:824] T ed5bfeedec294bd181422799ebda90ef P ee6ba208af054a77a67f4e0598cd8c63: Log is configured to *not* fsync() on all Append() calls
I20260929 10:35:46.749509 29652 heartbeater.cc:503] Master 127.15.82.126:37881 was elected leader, sending a full tablet report...
I20260929 10:35:46.749717 29938 ts_tablet_manager.cc:1428] T ed5bfeedec294bd181422799ebda90ef P c53d47f2877d47dc9e0e6d6793562c71: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260929 10:35:46.749763 29937 ts_tablet_manager.cc:1428] T ed5bfeedec294bd181422799ebda90ef P f379905b6ee84cc6bacf15584dcfd086: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260929 10:35:46.750056 29786 heartbeater.cc:503] Master 127.15.82.126:37881 was elected leader, sending a full tablet report...
I20260929 10:35:46.752053 29941 tablet_bootstrap.cc:492] T ed5bfeedec294bd181422799ebda90ef P ee6ba208af054a77a67f4e0598cd8c63: No bootstrap required, opened a new log
I20260929 10:35:46.752244 29941 ts_tablet_manager.cc:1397] T ed5bfeedec294bd181422799ebda90ef P ee6ba208af054a77a67f4e0598cd8c63: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260929 10:35:46.755538 29941 raft_consensus.cc:348] T ed5bfeedec294bd181422799ebda90ef P ee6ba208af054a77a67f4e0598cd8c63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f379905b6ee84cc6bacf15584dcfd086" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34003 } } peers { permanent_uuid: "ee6ba208af054a77a67f4e0598cd8c63" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 38555 } } peers { permanent_uuid: "c53d47f2877d47dc9e0e6d6793562c71" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 38923 } }
I20260929 10:35:46.755792 29941 raft_consensus.cc:374] T ed5bfeedec294bd181422799ebda90ef P ee6ba208af054a77a67f4e0598cd8c63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:35:46.755872 29941 raft_consensus.cc:729] T ed5bfeedec294bd181422799ebda90ef P ee6ba208af054a77a67f4e0598cd8c63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee6ba208af054a77a67f4e0598cd8c63, State: Initialized, Role: FOLLOWER
I20260929 10:35:46.756345 29941 consensus_queue.cc:260] T ed5bfeedec294bd181422799ebda90ef P ee6ba208af054a77a67f4e0598cd8c63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f379905b6ee84cc6bacf15584dcfd086" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34003 } } peers { permanent_uuid: "ee6ba208af054a77a67f4e0598cd8c63" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 38555 } } peers { permanent_uuid: "c53d47f2877d47dc9e0e6d6793562c71" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 38923 } }
I20260929 10:35:46.756927 29920 heartbeater.cc:503] Master 127.15.82.126:37881 was elected leader, sending a full tablet report...
I20260929 10:35:46.757179 29941 ts_tablet_manager.cc:1428] T ed5bfeedec294bd181422799ebda90ef P ee6ba208af054a77a67f4e0598cd8c63: Time spent starting tablet: real 0.005s user 0.001s sys 0.005s
W20260929 10:35:46.923369 29921 tablet.cc:2369] T ed5bfeedec294bd181422799ebda90ef P ee6ba208af054a77a67f4e0598cd8c63: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 10:35:46.944005 29787 tablet.cc:2369] T ed5bfeedec294bd181422799ebda90ef P c53d47f2877d47dc9e0e6d6793562c71: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 10:35:46.973649 29653 tablet.cc:2369] T ed5bfeedec294bd181422799ebda90ef P f379905b6ee84cc6bacf15584dcfd086: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 10:35:47.134905 29943 raft_consensus.cc:482] T ed5bfeedec294bd181422799ebda90ef P f379905b6ee84cc6bacf15584dcfd086 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:35:47.135166 29943 raft_consensus.cc:504] T ed5bfeedec294bd181422799ebda90ef P f379905b6ee84cc6bacf15584dcfd086 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f379905b6ee84cc6bacf15584dcfd086" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34003 } } peers { permanent_uuid: "ee6ba208af054a77a67f4e0598cd8c63" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 38555 } } peers { permanent_uuid: "c53d47f2877d47dc9e0e6d6793562c71" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 38923 } }
I20260929 10:35:47.136379 29943 leader_election.cc:288] T ed5bfeedec294bd181422799ebda90ef P f379905b6ee84cc6bacf15584dcfd086 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee6ba208af054a77a67f4e0598cd8c63 (127.15.82.67:38555), c53d47f2877d47dc9e0e6d6793562c71 (127.15.82.66:38923)
I20260929 10:35:47.142328 29936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:47.136922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.65:49265 (local address 127.15.82.66:38923)
0929 10:35:47.137046 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0929 10:35:47.137051 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:35:47.137393 (+ 342us) server_negotiation.cc:407] Connection header received
0929 10:35:47.137542 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:35:47.137546 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:35:47.137610 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:35:47.137711 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:35:47.138122 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:47.139079 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:47.140970 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:47.141639 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:47.141773 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:47.141836 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0929 10:35:47.142098 (+ 262us) server_negotiation.cc:299] Negotiation successful
0929 10:35:47.142179 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260929 10:35:47.142328 29935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:47.136843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.65:53169 (local address 127.15.82.67:38555)
0929 10:35:47.136979 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0929 10:35:47.136985 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 10:35:47.137393 (+ 408us) server_negotiation.cc:407] Connection header received
0929 10:35:47.137542 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:35:47.137546 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:35:47.137616 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:35:47.137710 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:35:47.138145 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:47.139074 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:47.140970 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:47.141625 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:47.141773 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:47.141836 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0929 10:35:47.142098 (+ 262us) server_negotiation.cc:299] Negotiation successful
0929 10:35:47.142179 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 10:35:47.142417 29946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:47.136771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.66:38923 (local address 127.15.82.65:49265)
0929 10:35:47.137281 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:47.137295 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 10:35:47.137415 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:35:47.137734 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:35:47.137737 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:35:47.137763 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:35:47.137971 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:47.137978 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:47.139215 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:47.139219 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:47.140772 (+ 1553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:47.140785 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:47.141784 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:47.141789 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:47.141934 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:47.141977 (+ 43us) client_negotiation.cc:769] Sending connection context
0929 10:35:47.142060 (+ 83us) client_negotiation.cc:240] Negotiation successful
0929 10:35:47.142187 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"mutex_wait_us":239,"spinlock_wait_cycles":20480,"thread_start_us":168,"threads_started":1}
I20260929 10:35:47.142422 29947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:47.136771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.67:38555 (local address 127.15.82.65:53169)
0929 10:35:47.137233 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:47.137249 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 10:35:47.137415 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:35:47.137734 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:35:47.137737 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:35:47.137763 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:35:47.137971 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:47.137978 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:47.139215 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:47.139219 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:47.140772 (+ 1553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:47.140785 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:47.141784 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:47.141789 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:47.141925 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:47.141983 (+ 58us) client_negotiation.cc:769] Sending connection context
0929 10:35:47.142060 (+ 77us) client_negotiation.cc:240] Negotiation successful
0929 10:35:47.142184 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":358,"mutex_wait_us":241,"spinlock_wait_cycles":20352,"thread_start_us":202,"threads_started":1}
I20260929 10:35:47.143206 29740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed5bfeedec294bd181422799ebda90ef" candidate_uuid: "f379905b6ee84cc6bacf15584dcfd086" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c53d47f2877d47dc9e0e6d6793562c71" is_pre_election: true
I20260929 10:35:47.143472 29874 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed5bfeedec294bd181422799ebda90ef" candidate_uuid: "f379905b6ee84cc6bacf15584dcfd086" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee6ba208af054a77a67f4e0598cd8c63" is_pre_election: true
I20260929 10:35:47.143551 29740 raft_consensus.cc:2454] T ed5bfeedec294bd181422799ebda90ef P c53d47f2877d47dc9e0e6d6793562c71 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f379905b6ee84cc6bacf15584dcfd086 in term 0.
I20260929 10:35:47.143766 29874 raft_consensus.cc:2454] T ed5bfeedec294bd181422799ebda90ef P ee6ba208af054a77a67f4e0598cd8c63 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f379905b6ee84cc6bacf15584dcfd086 in term 0.
I20260929 10:35:47.144068 29542 leader_election.cc:302] T ed5bfeedec294bd181422799ebda90ef P f379905b6ee84cc6bacf15584dcfd086 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c53d47f2877d47dc9e0e6d6793562c71, f379905b6ee84cc6bacf15584dcfd086; no voters:
I20260929 10:35:47.144404 29943 raft_consensus.cc:2781] T ed5bfeedec294bd181422799ebda90ef P f379905b6ee84cc6bacf15584dcfd086 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:35:47.144490 29943 raft_consensus.cc:482] T ed5bfeedec294bd181422799ebda90ef P f379905b6ee84cc6bacf15584dcfd086 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:35:47.144536 29943 raft_consensus.cc:3037] T ed5bfeedec294bd181422799ebda90ef P f379905b6ee84cc6bacf15584dcfd086 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:35:47.145656 29943 raft_consensus.cc:504] T ed5bfeedec294bd181422799ebda90ef P f379905b6ee84cc6bacf15584dcfd086 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f379905b6ee84cc6bacf15584dcfd086" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34003 } } peers { permanent_uuid: "ee6ba208af054a77a67f4e0598cd8c63" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 38555 } } peers { permanent_uuid: "c53d47f2877d47dc9e0e6d6793562c71" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 38923 } }
I20260929 10:35:47.146093 29943 leader_election.cc:288] T ed5bfeedec294bd181422799ebda90ef P f379905b6ee84cc6bacf15584dcfd086 [CANDIDATE]: Term 1 election: Requested vote from peers ee6ba208af054a77a67f4e0598cd8c63 (127.15.82.67:38555), c53d47f2877d47dc9e0e6d6793562c71 (127.15.82.66:38923)
I20260929 10:35:47.146539 29740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed5bfeedec294bd181422799ebda90ef" candidate_uuid: "f379905b6ee84cc6bacf15584dcfd086" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c53d47f2877d47dc9e0e6d6793562c71"
I20260929 10:35:47.146690 29740 raft_consensus.cc:3037] T ed5bfeedec294bd181422799ebda90ef P c53d47f2877d47dc9e0e6d6793562c71 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:35:47.146799 29874 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed5bfeedec294bd181422799ebda90ef" candidate_uuid: "f379905b6ee84cc6bacf15584dcfd086" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee6ba208af054a77a67f4e0598cd8c63"
I20260929 10:35:47.146912 29874 raft_consensus.cc:3037] T ed5bfeedec294bd181422799ebda90ef P ee6ba208af054a77a67f4e0598cd8c63 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:35:47.147933 29874 raft_consensus.cc:2454] T ed5bfeedec294bd181422799ebda90ef P ee6ba208af054a77a67f4e0598cd8c63 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f379905b6ee84cc6bacf15584dcfd086 in term 1.
I20260929 10:35:47.148205 29740 raft_consensus.cc:2454] T ed5bfeedec294bd181422799ebda90ef P c53d47f2877d47dc9e0e6d6793562c71 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f379905b6ee84cc6bacf15584dcfd086 in term 1.
I20260929 10:35:47.148459 29540 leader_election.cc:302] T ed5bfeedec294bd181422799ebda90ef P f379905b6ee84cc6bacf15584dcfd086 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ee6ba208af054a77a67f4e0598cd8c63, f379905b6ee84cc6bacf15584dcfd086; no voters:
I20260929 10:35:47.148713 29943 raft_consensus.cc:2781] T ed5bfeedec294bd181422799ebda90ef P f379905b6ee84cc6bacf15584dcfd086 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:35:47.148995 29943 raft_consensus.cc:686] T ed5bfeedec294bd181422799ebda90ef P f379905b6ee84cc6bacf15584dcfd086 [term 1 LEADER]: Becoming Leader. State: Replica: f379905b6ee84cc6bacf15584dcfd086, State: Running, Role: LEADER
I20260929 10:35:47.149346 29943 consensus_queue.cc:237] T ed5bfeedec294bd181422799ebda90ef P f379905b6ee84cc6bacf15584dcfd086 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f379905b6ee84cc6bacf15584dcfd086" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34003 } } peers { permanent_uuid: "ee6ba208af054a77a67f4e0598cd8c63" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 38555 } } peers { permanent_uuid: "c53d47f2877d47dc9e0e6d6793562c71" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 38923 } }
I20260929 10:35:47.153657 29460 catalog_manager.cc:5495] T ed5bfeedec294bd181422799ebda90ef P f379905b6ee84cc6bacf15584dcfd086 reported cstate change: term changed from 0 to 1, leader changed from <none> to f379905b6ee84cc6bacf15584dcfd086 (127.15.82.65). New cstate: current_term: 1 leader_uuid: "f379905b6ee84cc6bacf15584dcfd086" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f379905b6ee84cc6bacf15584dcfd086" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee6ba208af054a77a67f4e0598cd8c63" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 38555 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c53d47f2877d47dc9e0e6d6793562c71" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 38923 } health_report { overall_health: UNKNOWN } } }
I20260929 10:35:47.178577 29932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:47.173844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46032 (local address 127.15.82.65:34003)
0929 10:35:47.173978 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0929 10:35:47.173983 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:35:47.173996 (+ 13us) server_negotiation.cc:407] Connection header received
0929 10:35:47.174104 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:35:47.174108 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:35:47.174172 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:35:47.174291 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:35:47.174613 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:47.175417 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:47.176636 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:47.176989 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:47.177092 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:47.177445 (+ 353us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:35:47.178049 (+ 604us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:35:47.178182 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0929 10:35:47.178330 (+ 148us) server_negotiation.cc:299] Negotiation successful
0929 10:35:47.178427 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260929 10:35:47.186297 29740 raft_consensus.cc:1261] T ed5bfeedec294bd181422799ebda90ef P c53d47f2877d47dc9e0e6d6793562c71 [term 1 FOLLOWER]: Refusing update from remote peer f379905b6ee84cc6bacf15584dcfd086: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:35:47.186466 29874 raft_consensus.cc:1261] T ed5bfeedec294bd181422799ebda90ef P ee6ba208af054a77a67f4e0598cd8c63 [term 1 FOLLOWER]: Refusing update from remote peer f379905b6ee84cc6bacf15584dcfd086: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:35:47.187305 29943 consensus_queue.cc:1035] T ed5bfeedec294bd181422799ebda90ef P f379905b6ee84cc6bacf15584dcfd086 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee6ba208af054a77a67f4e0598cd8c63" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 38555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:35:47.187479 29948 consensus_queue.cc:1035] T ed5bfeedec294bd181422799ebda90ef P f379905b6ee84cc6bacf15584dcfd086 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c53d47f2877d47dc9e0e6d6793562c71" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 38923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:35:47.198983 29953 mvcc.cc:205] Tried to move back new op lower bound from 7334617690866782208 to 7334617690727698432. Current Snapshot: MvccSnapshot[applied={T|T < 7334617690866782208}]
I20260929 10:35:47.205731 29461 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47000:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:35:47.205890 29461 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47000:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:35:47.208571 29461 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 00ca51f69920468e9052d7591f5043ec: Sending DeleteTablet for 3 replicas of tablet ed5bfeedec294bd181422799ebda90ef
I20260929 10:35:47.209438 29586 tablet_service.cc:1514] Processing DeleteTablet for tablet ed5bfeedec294bd181422799ebda90ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:35:47 UTC) from {username='kudu', principal='kudu/127.15.82.126@KRBTEST.COM'} at 127.0.0.1:46022
I20260929 10:35:47.209932 29961 tablet_replica.cc:330] T ed5bfeedec294bd181422799ebda90ef P f379905b6ee84cc6bacf15584dcfd086: stopping tablet replica
I20260929 10:35:47.209981 29854 tablet_service.cc:1514] Processing DeleteTablet for tablet ed5bfeedec294bd181422799ebda90ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:35:47 UTC) from {username='kudu', principal='kudu/127.15.82.126@KRBTEST.COM'} at 127.0.0.1:54412
I20260929 10:35:47.210428 29961 raft_consensus.cc:2229] T ed5bfeedec294bd181422799ebda90ef P f379905b6ee84cc6bacf15584dcfd086 [term 1 LEADER]: Raft consensus shutting down.
I20260929 10:35:47.210947 29961 raft_consensus.cc:2258] T ed5bfeedec294bd181422799ebda90ef P f379905b6ee84cc6bacf15584dcfd086 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:35:47.211072 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 29523
I20260929 10:35:47.211005 29720 tablet_service.cc:1514] Processing DeleteTablet for tablet ed5bfeedec294bd181422799ebda90ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:35:47 UTC) from {username='kudu', principal='kudu/127.15.82.126@KRBTEST.COM'} at 127.0.0.1:53770
I20260929 10:35:47.215107 29963 tablet_replica.cc:330] T ed5bfeedec294bd181422799ebda90ef P c53d47f2877d47dc9e0e6d6793562c71: stopping tablet replica
I20260929 10:35:47.215366 29963 raft_consensus.cc:2229] T ed5bfeedec294bd181422799ebda90ef P c53d47f2877d47dc9e0e6d6793562c71 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:35:47.215636 29963 raft_consensus.cc:2258] T ed5bfeedec294bd181422799ebda90ef P c53d47f2877d47dc9e0e6d6793562c71 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:35:47.215989 29962 tablet_replica.cc:330] T ed5bfeedec294bd181422799ebda90ef P ee6ba208af054a77a67f4e0598cd8c63: stopping tablet replica
I20260929 10:35:47.216296 29962 raft_consensus.cc:2229] T ed5bfeedec294bd181422799ebda90ef P ee6ba208af054a77a67f4e0598cd8c63 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:35:47.216612 29962 raft_consensus.cc:2258] T ed5bfeedec294bd181422799ebda90ef P ee6ba208af054a77a67f4e0598cd8c63 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:35:47.217337 29963 ts_tablet_manager.cc:1905] T ed5bfeedec294bd181422799ebda90ef P c53d47f2877d47dc9e0e6d6793562c71: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:35:47.218034 29962 ts_tablet_manager.cc:1905] T ed5bfeedec294bd181422799ebda90ef P ee6ba208af054a77a67f4e0598cd8c63: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260929 10:35:47.219269 29445 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.82.65:34003 (error 108)
I20260929 10:35:47.220074 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 29657
I20260929 10:35:47.220299 29934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:47.219993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.65:34003 (local address 127.0.0.1:46042)
0929 10:35:47.220130 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:47.220198 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.65:34003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260929 10:35:47.220749 29445 catalog_manager.cc:4584] TS f379905b6ee84cc6bacf15584dcfd086 (127.15.82.65:34003): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ed5bfeedec294bd181422799ebda90ef: Network error: Client connection negotiation failed: client connection to 127.15.82.65:34003: connect: Connection refused (error 111)
I20260929 10:35:47.221594 29962 ts_tablet_manager.cc:1918] T ed5bfeedec294bd181422799ebda90ef P ee6ba208af054a77a67f4e0598cd8c63: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:35:47.221697 29962 log.cc:1196] T ed5bfeedec294bd181422799ebda90ef P ee6ba208af054a77a67f4e0598cd8c63: Deleting WAL directory at /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-2/wal/wals/ed5bfeedec294bd181422799ebda90ef
I20260929 10:35:47.222031 29962 ts_tablet_manager.cc:1939] T ed5bfeedec294bd181422799ebda90ef P ee6ba208af054a77a67f4e0598cd8c63: Deleting consensus metadata
I20260929 10:35:47.223353 29446 catalog_manager.cc:4841] TS ee6ba208af054a77a67f4e0598cd8c63 (127.15.82.67:38555): tablet ed5bfeedec294bd181422799ebda90ef (table test-table [id=eea4cb7e40544a5ab5d1a3ebc0ec82ef]) successfully deleted
I20260929 10:35:47.227520 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 29791
I20260929 10:35:47.227880 29934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:47.227576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.66:38923 (local address 127.0.0.1:53780)
0929 10:35:47.227724 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:47.227784 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.66:38923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260929 10:35:47.233587 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 29429
I20260929 10:35:47.240784 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:37881
--webserver_interface=127.15.82.126
--webserver_port=45403
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab
--principal=kudu/127.15.82.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:44075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.82.126:37881
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:35:47.354988 29965 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:35:47.355273 29965 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:35:47.355317 29965 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:35:47.359578 29965 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:35:47.359643 29965 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:35:47.359666 29965 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:35:47.359686 29965 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:35:47.359745 29965 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:35:47.365535 29965 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:44075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.15.82.126:37881
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab
--principal=kudu/127.15.82.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:37881
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.82.126
--webserver_port=45403
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29965
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:35:47.366731 29965 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:35:47.367796 29965 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 10:35:47.381593 29965 server_base.cc:1034] running on GCE node
W20260929 10:35:47.381565 29974 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:35:47.381783 29971 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:35:47.381594 29972 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:35:47.382553 29965 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:35:47.383769 29965 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:35:47.384981 29965 hybrid_clock.cc:648] HybridClock initialized: now 1790678147384951 us; error 47 us; skew 500 ppm
Sep 29 10:35:47 dist-test-slave-vg89 krb5kdc[29416](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678147, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:35:47.388226 29965 init.cc:377] Logged in from keytab as kudu/127.15.82.126@KRBTEST.COM (short username kudu)
I20260929 10:35:47.389195 29965 webserver.cc:460] Webserver started at http://127.15.82.126:45403/ using document root <none> and password file <none>
I20260929 10:35:47.389911 29965 fs_manager.cc:359] Metadata directory not provided
I20260929 10:35:47.390021 29965 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:35:47.394361 29965 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260929 10:35:47.396744 29981 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:35:47.398140 29965 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 10:35:47.398303 29965 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/master-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/master-0/wal
uuid: "00ca51f69920468e9052d7591f5043ec"
format_stamp: "Formatted at 2026-09-29 10:35:45 on dist-test-slave-vg89"
server_key: "70936bd37ac113f4769fa6afe351f761"
server_key_iv: "a921de9c6f3d8ac2d245aab071461444"
server_key_version: "encryptionkey@0"
I20260929 10:35:47.399019 29965 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:35:47.433107 29965 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:35:47.436129 29965 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:35:47.436354 29965 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:35:47.443558 29965 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.126:37881
I20260929 10:35:47.443850 30033 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.126:37881 every 8 connection(s)
I20260929 10:35:47.444746 29965 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
I20260929 10:35:47.448283 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 29965
I20260929 10:35:47.448752 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.65:34003
--local_ip_for_outbound_sockets=127.15.82.65
--tserver_master_addrs=127.15.82.126:37881
--webserver_port=35431
--webserver_interface=127.15.82.65
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab
--principal=kudu/127.15.82.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:44075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 10:35:47.449347 30034 sys_catalog.cc:263] Verifying existing consensus state
I20260929 10:35:47.451438 30034 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00ca51f69920468e9052d7591f5043ec: Bootstrap starting.
I20260929 10:35:47.468451 30034 log.cc:824] T 00000000000000000000000000000000 P 00ca51f69920468e9052d7591f5043ec: Log is configured to *not* fsync() on all Append() calls
I20260929 10:35:47.477063 30034 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00ca51f69920468e9052d7591f5043ec: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 10:35:47.477453 30034 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00ca51f69920468e9052d7591f5043ec: Bootstrap complete.
I20260929 10:35:47.481541 30034 raft_consensus.cc:348] T 00000000000000000000000000000000 P 00ca51f69920468e9052d7591f5043ec [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ca51f69920468e9052d7591f5043ec" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 37881 } }
I20260929 10:35:47.481968 30034 raft_consensus.cc:729] T 00000000000000000000000000000000 P 00ca51f69920468e9052d7591f5043ec [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00ca51f69920468e9052d7591f5043ec, State: Initialized, Role: FOLLOWER
I20260929 10:35:47.482504 30034 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00ca51f69920468e9052d7591f5043ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ca51f69920468e9052d7591f5043ec" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 37881 } }
I20260929 10:35:47.482649 30034 raft_consensus.cc:388] T 00000000000000000000000000000000 P 00ca51f69920468e9052d7591f5043ec [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:35:47.482749 30034 raft_consensus.cc:482] T 00000000000000000000000000000000 P 00ca51f69920468e9052d7591f5043ec [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:35:47.482862 30034 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 00ca51f69920468e9052d7591f5043ec [term 1 FOLLOWER]: Advancing to term 2
I20260929 10:35:47.484704 30034 raft_consensus.cc:504] T 00000000000000000000000000000000 P 00ca51f69920468e9052d7591f5043ec [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ca51f69920468e9052d7591f5043ec" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 37881 } }
I20260929 10:35:47.485046 30034 leader_election.cc:302] T 00000000000000000000000000000000 P 00ca51f69920468e9052d7591f5043ec [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 00ca51f69920468e9052d7591f5043ec; no voters:
I20260929 10:35:47.485376 30034 leader_election.cc:288] T 00000000000000000000000000000000 P 00ca51f69920468e9052d7591f5043ec [CANDIDATE]: Term 2 election: Requested vote from peers
I20260929 10:35:47.485467 30038 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 00ca51f69920468e9052d7591f5043ec [term 2 FOLLOWER]: Leader election won for term 2
I20260929 10:35:47.485666 30038 raft_consensus.cc:686] T 00000000000000000000000000000000 P 00ca51f69920468e9052d7591f5043ec [term 2 LEADER]: Becoming Leader. State: Replica: 00ca51f69920468e9052d7591f5043ec, State: Running, Role: LEADER
I20260929 10:35:47.485950 30038 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00ca51f69920468e9052d7591f5043ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ca51f69920468e9052d7591f5043ec" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 37881 } }
I20260929 10:35:47.486624 30034 sys_catalog.cc:564] T 00000000000000000000000000000000 P 00ca51f69920468e9052d7591f5043ec [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:35:47.487263 30039 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00ca51f69920468e9052d7591f5043ec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "00ca51f69920468e9052d7591f5043ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ca51f69920468e9052d7591f5043ec" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 37881 } } }
I20260929 10:35:47.487388 30039 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00ca51f69920468e9052d7591f5043ec [sys.catalog]: This master's current role is: LEADER
I20260929 10:35:47.488032 30045 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:35:47.487994 30040 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00ca51f69920468e9052d7591f5043ec [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00ca51f69920468e9052d7591f5043ec. Latest consensus state: current_term: 2 leader_uuid: "00ca51f69920468e9052d7591f5043ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ca51f69920468e9052d7591f5043ec" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 37881 } } }
I20260929 10:35:47.488211 30040 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00ca51f69920468e9052d7591f5043ec [sys.catalog]: This master's current role is: LEADER
I20260929 10:35:47.492983 30045 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:35:47.493880 30045 catalog_manager.cc:1259] Loaded cluster ID: d070de294db6467bb3d6ee46974440f3
I20260929 10:35:47.493940 30045 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:35:47.495689 30045 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260929 10:35:47.500126 30045 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260929 10:35:47.505703 30045 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:35:47.506498 30045 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 00ca51f69920468e9052d7591f5043ec: Loaded TSK: 0
I20260929 10:35:47.507663 30045 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260929 10:35:47.575501 30036 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:35:47.576043 30036 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:35:47.576406 30036 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:35:47.584944 30036 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:35:47.585036 30036 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:35:47.585167 30036 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.65
I20260929 10:35:47.594272 30036 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:44075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab
--principal=kudu/127.15.82.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.65:34003
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.82.65
--webserver_port=35431
--tserver_master_addrs=127.15.82.126:37881
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30036
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.65
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:35:47.595618 30036 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:35:47.596946 30036 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:35:47.606920 30061 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:35:47.606891 30062 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:35:47.607112 30036 server_base.cc:1034] running on GCE node
W20260929 10:35:47.606912 30064 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:35:47.607640 30036 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:35:47.608377 30036 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:35:47.609591 30036 hybrid_clock.cc:648] HybridClock initialized: now 1790678147609572 us; error 34 us; skew 500 ppm
Sep 29 10:35:47 dist-test-slave-vg89 krb5kdc[29416](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678147, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:35:47.612561 30036 init.cc:377] Logged in from keytab as kudu/127.15.82.65@KRBTEST.COM (short username kudu)
I20260929 10:35:47.613554 30036 webserver.cc:460] Webserver started at http://127.15.82.65:35431/ using document root <none> and password file <none>
I20260929 10:35:47.614135 30036 fs_manager.cc:359] Metadata directory not provided
I20260929 10:35:47.614243 30036 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:35:47.618381 30036 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.003s
I20260929 10:35:47.620896 30071 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:35:47.622275 30036 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260929 10:35:47.622471 30036 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-0/wal
uuid: "f379905b6ee84cc6bacf15584dcfd086"
format_stamp: "Formatted at 2026-09-29 10:35:46 on dist-test-slave-vg89"
server_key: "412b7efc5455c0d3d9eb9b84117cdc25"
server_key_iv: "3d151e1d30376524fe048ce203a8084e"
server_key_version: "encryptionkey@0"
I20260929 10:35:47.623090 30036 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:35:47.636294 30036 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:35:47.639782 30036 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:35:47.639998 30036 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:35:47.640667 30036 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:35:47.641837 30078 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260929 10:35:47.647087 30036 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260929 10:35:47.647189 30036 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260929 10:35:47.647279 30036 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260929 10:35:47.648960 30036 ts_tablet_manager.cc:610] Registered 1 tablets
I20260929 10:35:47.649029 30036 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260929 10:35:47.649302 30078 tablet_bootstrap.cc:492] T ed5bfeedec294bd181422799ebda90ef P f379905b6ee84cc6bacf15584dcfd086: Bootstrap starting.
I20260929 10:35:47.659605 30036 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.65:34003
I20260929 10:35:47.659653 30185 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.65:34003 every 8 connection(s)
I20260929 10:35:47.660660 30036 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
I20260929 10:35:47.669678 30078 log.cc:824] T ed5bfeedec294bd181422799ebda90ef P f379905b6ee84cc6bacf15584dcfd086: Log is configured to *not* fsync() on all Append() calls
I20260929 10:35:47.670097 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 30036
I20260929 10:35:47.670620 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.66:38923
--local_ip_for_outbound_sockets=127.15.82.66
--tserver_master_addrs=127.15.82.126:37881
--webserver_port=40525
--webserver_interface=127.15.82.66
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab
--principal=kudu/127.15.82.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:44075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 10:35:47 dist-test-slave-vg89 krb5kdc[29416](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678147, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.65@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:35:47.678704 30188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:47.662690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.65:41929 (local address 127.15.82.126:37881)
0929 10:35:47.663106 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0929 10:35:47.663132 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0929 10:35:47.663414 (+ 282us) server_negotiation.cc:407] Connection header received
0929 10:35:47.664605 (+ 1191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:35:47.664624 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:35:47.665082 (+ 458us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:35:47.665452 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:35:47.666629 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:47.668281 (+ 1652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:47.669551 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:47.670302 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:47.670529 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:47.673399 (+ 2870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:35:47.673405 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:35:47.673465 (+ 60us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:35:47.673502 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:35:47.675487 (+ 1985us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:35:47.676025 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:35:47.676030 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:35:47.676033 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:35:47.676146 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:35:47.676516 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:35:47.676519 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:35:47.676520 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:35:47.676852 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:35:47.677004 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0929 10:35:47.677539 (+ 535us) server_negotiation.cc:299] Negotiation successful
0929 10:35:47.677794 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":239,"thread_start_us":109,"threads_started":1}
I20260929 10:35:47.678714 30189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:47.662676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:37881 (local address 127.15.82.65:41929)
0929 10:35:47.663228 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:47.663280 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0929 10:35:47.664101 (+ 821us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:35:47.665522 (+ 1421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:35:47.665529 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:35:47.665878 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:35:47.666367 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:47.666380 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:47.668430 (+ 2050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:47.668433 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:47.669406 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:47.669424 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:47.670483 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:47.670489 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:47.670688 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:47.671341 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:35:47.671362 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:35:47.673187 (+ 1825us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:35:47.675611 (+ 2424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:35:47.675617 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:35:47.675630 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:35:47.675892 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:35:47.676246 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:35:47.676250 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:35:47.676252 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:35:47.676393 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:35:47.676988 (+ 595us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:35:47.676994 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:35:47.677231 (+ 237us) client_negotiation.cc:769] Sending connection context
0929 10:35:47.677416 (+ 185us) client_negotiation.cc:240] Negotiation successful
0929 10:35:47.677755 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":137,"threads_started":1}
I20260929 10:35:47.679342 30078 tablet_bootstrap.cc:492] T ed5bfeedec294bd181422799ebda90ef P f379905b6ee84cc6bacf15584dcfd086: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 10:35:47.679857 30078 tablet_bootstrap.cc:492] T ed5bfeedec294bd181422799ebda90ef P f379905b6ee84cc6bacf15584dcfd086: Bootstrap complete.
I20260929 10:35:47.681222 30186 heartbeater.cc:347] Connected to a master server at 127.15.82.126:37881
I20260929 10:35:47.681391 30078 ts_tablet_manager.cc:1397] T ed5bfeedec294bd181422799ebda90ef P f379905b6ee84cc6bacf15584dcfd086: Time spent bootstrapping tablet: real 0.032s user 0.030s sys 0.000s
I20260929 10:35:47.681476 30186 heartbeater.cc:464] Registering TS with master...
I20260929 10:35:47.682101 30186 heartbeater.cc:511] Master 127.15.82.126:37881 requested a full tablet report, sending...
I20260929 10:35:47.684665 29998 ts_manager.cc:194] Registered new tserver with Master: f379905b6ee84cc6bacf15584dcfd086 (127.15.82.65:34003)
I20260929 10:35:47.684908 30078 raft_consensus.cc:348] T ed5bfeedec294bd181422799ebda90ef P f379905b6ee84cc6bacf15584dcfd086 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f379905b6ee84cc6bacf15584dcfd086" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34003 } } peers { permanent_uuid: "ee6ba208af054a77a67f4e0598cd8c63" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 38555 } } peers { permanent_uuid: "c53d47f2877d47dc9e0e6d6793562c71" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 38923 } }
I20260929 10:35:47.685446 30078 raft_consensus.cc:729] T ed5bfeedec294bd181422799ebda90ef P f379905b6ee84cc6bacf15584dcfd086 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f379905b6ee84cc6bacf15584dcfd086, State: Initialized, Role: FOLLOWER
I20260929 10:35:47.685972 30078 consensus_queue.cc:260] T ed5bfeedec294bd181422799ebda90ef P f379905b6ee84cc6bacf15584dcfd086 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f379905b6ee84cc6bacf15584dcfd086" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34003 } } peers { permanent_uuid: "ee6ba208af054a77a67f4e0598cd8c63" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 38555 } } peers { permanent_uuid: "c53d47f2877d47dc9e0e6d6793562c71" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 38923 } }
I20260929 10:35:47.687050 30078 ts_tablet_manager.cc:1428] T ed5bfeedec294bd181422799ebda90ef P f379905b6ee84cc6bacf15584dcfd086: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260929 10:35:47.689049 29998 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.82.65@KRBTEST.COM'} at 127.15.82.65:41929
I20260929 10:35:47.691485 30186 heartbeater.cc:503] Master 127.15.82.126:37881 was elected leader, sending a full tablet report...
Sep 29 10:35:47 dist-test-slave-vg89 krb5kdc[29416](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678147, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.126@KRBTEST.COM for kudu/127.15.82.65@KRBTEST.COM
I20260929 10:35:47.699652 30193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:47.687951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.65:34003 (local address 127.0.0.1:46048)
0929 10:35:47.688744 (+ 793us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:47.688788 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 10:35:47.688942 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:35:47.689487 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:35:47.689493 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:35:47.689923 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:35:47.690059 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:47.690066 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:47.691587 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:47.691591 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:47.692410 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:47.692419 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:47.693020 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:47.693023 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:47.693136 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:47.693681 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:35:47.693695 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:35:47.695514 (+ 1819us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:35:47.697718 (+ 2204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:35:47.697726 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:35:47.697730 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:35:47.698020 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:35:47.698322 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:35:47.698324 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:35:47.698325 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:35:47.698536 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:35:47.699084 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:35:47.699089 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:35:47.699203 (+ 114us) client_negotiation.cc:769] Sending connection context
0929 10:35:47.699333 (+ 130us) client_negotiation.cc:240] Negotiation successful
0929 10:35:47.699513 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":617,"thread_start_us":114,"threads_started":1}
I20260929 10:35:47.699770 30194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:47.688526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46048 (local address 127.15.82.65:34003)
0929 10:35:47.688808 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0929 10:35:47.688818 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 10:35:47.688864 (+ 46us) server_negotiation.cc:407] Connection header received
0929 10:35:47.689098 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:35:47.689105 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:35:47.689313 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:35:47.689476 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:35:47.690195 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:47.691454 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:47.692542 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:47.692879 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:47.693005 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:47.695660 (+ 2655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:35:47.695669 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:35:47.695683 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:35:47.695707 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:35:47.697594 (+ 1887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:35:47.698132 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:35:47.698137 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:35:47.698139 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:35:47.698229 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:35:47.698653 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:35:47.698656 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:35:47.698658 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:35:47.698955 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:35:47.699087 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0929 10:35:47.699432 (+ 345us) server_negotiation.cc:299] Negotiation successful
0929 10:35:47.699600 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":119,"threads_started":1}
I20260929 10:35:47.701098 30120 tablet_service.cc:1514] Processing DeleteTablet for tablet ed5bfeedec294bd181422799ebda90ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:35:47 UTC) from {username='kudu', principal='kudu/127.15.82.126@KRBTEST.COM'} at 127.0.0.1:46048
I20260929 10:35:47.701978 30195 tablet_replica.cc:330] stopping tablet replica
I20260929 10:35:47.702342 30195 raft_consensus.cc:2229] T ed5bfeedec294bd181422799ebda90ef P f379905b6ee84cc6bacf15584dcfd086 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:35:47.702682 30195 raft_consensus.cc:2258] T ed5bfeedec294bd181422799ebda90ef P f379905b6ee84cc6bacf15584dcfd086 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:35:47.703792 30195 ts_tablet_manager.cc:1905] T ed5bfeedec294bd181422799ebda90ef P f379905b6ee84cc6bacf15584dcfd086: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:35:47.707572 30195 ts_tablet_manager.cc:1918] T ed5bfeedec294bd181422799ebda90ef P f379905b6ee84cc6bacf15584dcfd086: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:35:47.707700 30195 log.cc:1196] T ed5bfeedec294bd181422799ebda90ef P f379905b6ee84cc6bacf15584dcfd086: Deleting WAL directory at /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-0/wal/wals/ed5bfeedec294bd181422799ebda90ef
I20260929 10:35:47.708006 30195 ts_tablet_manager.cc:1939] T ed5bfeedec294bd181422799ebda90ef P f379905b6ee84cc6bacf15584dcfd086: Deleting consensus metadata
I20260929 10:35:47.709794 29982 catalog_manager.cc:4841] TS f379905b6ee84cc6bacf15584dcfd086 (127.15.82.65:34003): tablet ed5bfeedec294bd181422799ebda90ef (table test-table [id=eea4cb7e40544a5ab5d1a3ebc0ec82ef]) successfully deleted
W20260929 10:35:47.791605 30191 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:35:47.791899 30191 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:35:47.791944 30191 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:35:47.796110 30191 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:35:47.796177 30191 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:35:47.796241 30191 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.66
I20260929 10:35:47.801687 30191 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:44075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab
--principal=kudu/127.15.82.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.66:38923
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.82.66
--webserver_port=40525
--tserver_master_addrs=127.15.82.126:37881
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30191
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.66
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:35:47.802868 30191 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:35:47.804013 30191 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:35:47.813614 30201 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:35:47.813663 30200 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:35:47.813614 30203 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:35:47.813971 30191 server_base.cc:1034] running on GCE node
I20260929 10:35:47.814455 30191 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:35:47.815127 30191 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:35:47.816288 30191 hybrid_clock.cc:648] HybridClock initialized: now 1790678147816270 us; error 30 us; skew 500 ppm
Sep 29 10:35:47 dist-test-slave-vg89 krb5kdc[29416](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678147, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:35:47.819121 30191 init.cc:377] Logged in from keytab as kudu/127.15.82.66@KRBTEST.COM (short username kudu)
I20260929 10:35:47.819984 30191 webserver.cc:460] Webserver started at http://127.15.82.66:40525/ using document root <none> and password file <none>
I20260929 10:35:47.820574 30191 fs_manager.cc:359] Metadata directory not provided
I20260929 10:35:47.820686 30191 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:35:47.824635 30191 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260929 10:35:47.826931 30210 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:35:47.828469 30191 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 10:35:47.828604 30191 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-1/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-1/wal
uuid: "c53d47f2877d47dc9e0e6d6793562c71"
format_stamp: "Formatted at 2026-09-29 10:35:46 on dist-test-slave-vg89"
server_key: "fa1f3b5e7b477f15c03b5a1d21684acc"
server_key_iv: "0ca6a0f33ea9194c85e94ea0e0e92a63"
server_key_version: "encryptionkey@0"
I20260929 10:35:47.829285 30191 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:35:47.847842 30191 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:35:47.851258 30191 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:35:47.851476 30191 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:35:47.852123 30191 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:35:47.853296 30217 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
W20260929 10:35:47.857617 30217 ts_tablet_manager.cc:1852] T ed5bfeedec294bd181422799ebda90ef P c53d47f2877d47dc9e0e6d6793562c71: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260929 10:35:47.858119 30217 ts_tablet_manager.cc:1905] T ed5bfeedec294bd181422799ebda90ef P c53d47f2877d47dc9e0e6d6793562c71: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:35:47.862566 30217 ts_tablet_manager.cc:1918] T ed5bfeedec294bd181422799ebda90ef P c53d47f2877d47dc9e0e6d6793562c71: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:35:47.862692 30217 log.cc:1196] T ed5bfeedec294bd181422799ebda90ef P c53d47f2877d47dc9e0e6d6793562c71: Deleting WAL directory at /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-1/wal/wals/ed5bfeedec294bd181422799ebda90ef
I20260929 10:35:47.863149 30217 ts_tablet_manager.cc:1939] T ed5bfeedec294bd181422799ebda90ef P c53d47f2877d47dc9e0e6d6793562c71: Deleting consensus metadata
I20260929 10:35:47.863534 30191 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260929 10:35:47.863691 30191 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.011s user 0.000s sys 0.000s
I20260929 10:35:47.863791 30191 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:35:47.863847 30191 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:35:47.873986 30191 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.66:38923
I20260929 10:35:47.874014 30324 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.66:38923 every 8 connection(s)
I20260929 10:35:47.875062 30191 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
I20260929 10:35:47.878708 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 30191
I20260929 10:35:47.879194 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.67:38555
--local_ip_for_outbound_sockets=127.15.82.67
--tserver_master_addrs=127.15.82.126:37881
--webserver_port=37299
--webserver_interface=127.15.82.67
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab
--principal=kudu/127.15.82.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:44075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 10:35:47 dist-test-slave-vg89 krb5kdc[29416](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678147, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.66@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:35:47.892622 30188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:47.877134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:36571 (local address 127.15.82.126:37881)
0929 10:35:47.877289 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0929 10:35:47.877293 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:35:47.878309 (+ 1016us) server_negotiation.cc:407] Connection header received
0929 10:35:47.879764 (+ 1455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:35:47.879767 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:35:47.879824 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:35:47.879901 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:35:47.881552 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:47.882432 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:47.883593 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:47.884060 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:47.884197 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:47.887188 (+ 2991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:35:47.887196 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:35:47.887199 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:35:47.887239 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:35:47.889274 (+ 2035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:35:47.890104 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:35:47.890111 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:35:47.890113 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:35:47.890237 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:35:47.890931 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:35:47.890935 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:35:47.890937 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:35:47.891207 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:35:47.891317 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0929 10:35:47.892348 (+ 1031us) server_negotiation.cc:299] Negotiation successful
0929 10:35:47.892483 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260929 10:35:47.893744 30327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:47.877572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:37881 (local address 127.15.82.66:36571)
0929 10:35:47.878117 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:47.878166 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0929 10:35:47.879510 (+ 1344us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:35:47.880139 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:35:47.880151 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:35:47.880695 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:35:47.881375 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:47.881393 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:47.882560 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:47.882563 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:47.883448 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:47.883458 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:47.884231 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:47.884234 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:47.884387 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:47.885072 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:35:47.885089 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:35:47.886989 (+ 1900us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:35:47.889433 (+ 2444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:35:47.889463 (+ 30us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:35:47.889487 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:35:47.889941 (+ 454us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:35:47.890364 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:35:47.890370 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:35:47.890374 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:35:47.890746 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:35:47.891359 (+ 613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:35:47.891370 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:35:47.892040 (+ 670us) client_negotiation.cc:769] Sending connection context
0929 10:35:47.892269 (+ 229us) client_negotiation.cc:240] Negotiation successful
0929 10:35:47.892651 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":120,"threads_started":1}
I20260929 10:35:47.895076 30325 heartbeater.cc:347] Connected to a master server at 127.15.82.126:37881
I20260929 10:35:47.895409 30325 heartbeater.cc:464] Registering TS with master...
I20260929 10:35:47.896029 30325 heartbeater.cc:511] Master 127.15.82.126:37881 requested a full tablet report, sending...
I20260929 10:35:47.897210 29998 ts_manager.cc:194] Registered new tserver with Master: c53d47f2877d47dc9e0e6d6793562c71 (127.15.82.66:38923)
I20260929 10:35:47.897826 29998 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.82.66@KRBTEST.COM'} at 127.15.82.66:36571
W20260929 10:35:48.000659 30328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:35:48.000974 30328 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:35:48.001020 30328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:35:48.005602 30328 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:35:48.005677 30328 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:35:48.005769 30328 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.67
I20260929 10:35:48.011619 30328 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:44075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab
--principal=kudu/127.15.82.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.67:38555
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.82.67
--webserver_port=37299
--tserver_master_addrs=127.15.82.126:37881
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30328
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.67
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:35:48.012732 30328 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:35:48.013767 30328 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:35:48.023442 30336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:35:48.023419 30333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:35:48.023483 30334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:35:48.023640 30328 server_base.cc:1034] running on GCE node
I20260929 10:35:48.024117 30328 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:35:48.024732 30328 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:35:48.025889 30328 hybrid_clock.cc:648] HybridClock initialized: now 1790678148025871 us; error 28 us; skew 500 ppm
Sep 29 10:35:48 dist-test-slave-vg89 krb5kdc[29416](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678148, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:35:48.028756 30328 init.cc:377] Logged in from keytab as kudu/127.15.82.67@KRBTEST.COM (short username kudu)
I20260929 10:35:48.029613 30328 webserver.cc:460] Webserver started at http://127.15.82.67:37299/ using document root <none> and password file <none>
I20260929 10:35:48.030176 30328 fs_manager.cc:359] Metadata directory not provided
I20260929 10:35:48.030269 30328 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:35:48.034340 30328 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260929 10:35:48.036612 30343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:35:48.037910 30328 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260929 10:35:48.038005 30328 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-2/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-2/wal
uuid: "ee6ba208af054a77a67f4e0598cd8c63"
format_stamp: "Formatted at 2026-09-29 10:35:46 on dist-test-slave-vg89"
server_key: "3ecac6d030922b70a081b8bd106edd1c"
server_key_iv: "49980e6545024fe31ced90f03bc88c82"
server_key_version: "encryptionkey@0"
I20260929 10:35:48.038653 30328 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:35:48.079970 30328 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:35:48.083376 30328 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:35:48.083601 30328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:35:48.084266 30328 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:35:48.085139 30328 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:35:48.085188 30328 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:35:48.085254 30328 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:35:48.085287 30328 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:35:48.095155 30328 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.67:38555
I20260929 10:35:48.095177 30456 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.67:38555 every 8 connection(s)
I20260929 10:35:48.096312 30328 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
I20260929 10:35:48.097723 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 30328
Sep 29 10:35:48 dist-test-slave-vg89 krb5kdc[29416](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678148, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.67@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:35:48.107277 30460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:48.098423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47004 (local address 127.15.82.126:37881)
0929 10:35:48.098819 (+ 396us) server_negotiation.cc:206] Beginning negotiation
0929 10:35:48.098822 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:35:48.098878 (+ 56us) server_negotiation.cc:407] Connection header received
0929 10:35:48.099043 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:35:48.099046 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:35:48.099087 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:35:48.099167 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:35:48.100154 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:48.101070 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:48.101933 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:48.102191 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:48.102303 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:48.103813 (+ 1510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:35:48.103817 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:35:48.103820 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:35:48.103850 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:35:48.105410 (+ 1560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:35:48.105962 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:35:48.105965 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:35:48.105967 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:35:48.106055 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:35:48.106348 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:35:48.106350 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:35:48.106351 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:35:48.106590 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:35:48.106811 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0929 10:35:48.106958 (+ 147us) server_negotiation.cc:299] Negotiation successful
0929 10:35:48.107087 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":69,"server-negotiator.queue_time_us":332,"thread_start_us":100,"threads_started":1}
I20260929 10:35:48.111097 30188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:48.098416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.67:60831 (local address 127.15.82.126:37881)
0929 10:35:48.098554 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0929 10:35:48.098559 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:35:48.099644 (+ 1085us) server_negotiation.cc:407] Connection header received
0929 10:35:48.101353 (+ 1709us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:35:48.101356 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:35:48.101417 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:35:48.101515 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:35:48.102982 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:48.103538 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:48.104687 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:48.104915 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:48.105025 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:48.107783 (+ 2758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:35:48.107787 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:35:48.107789 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:35:48.107812 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:35:48.109286 (+ 1474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:35:48.109735 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:35:48.109738 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:35:48.109739 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:35:48.109804 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:35:48.110120 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:35:48.110122 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:35:48.110124 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:35:48.110271 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:35:48.110339 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0929 10:35:48.110874 (+ 535us) server_negotiation.cc:299] Negotiation successful
0929 10:35:48.110964 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260929 10:35:48.111958 30461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:48.098962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:37881 (local address 127.15.82.67:60831)
0929 10:35:48.099502 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:48.099544 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 10:35:48.101122 (+ 1578us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:35:48.101704 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:35:48.101711 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:35:48.102187 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:35:48.102822 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:48.102838 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:48.103663 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:48.103667 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:48.104560 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:48.104569 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:48.105066 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:48.105070 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:48.105249 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:48.105766 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:35:48.105789 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:35:48.107615 (+ 1826us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:35:48.109382 (+ 1767us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:35:48.109387 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:35:48.109399 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:35:48.109646 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:35:48.109910 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:35:48.109912 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:35:48.109913 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:35:48.110040 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:35:48.110352 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:35:48.110356 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:35:48.110660 (+ 304us) client_negotiation.cc:769] Sending connection context
0929 10:35:48.110814 (+ 154us) client_negotiation.cc:240] Negotiation successful
0929 10:35:48.111104 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":144,"threads_started":1}
I20260929 10:35:48.113049 30457 heartbeater.cc:347] Connected to a master server at 127.15.82.126:37881
I20260929 10:35:48.113299 30457 heartbeater.cc:464] Registering TS with master...
I20260929 10:35:48.113795 30457 heartbeater.cc:511] Master 127.15.82.126:37881 requested a full tablet report, sending...
I20260929 10:35:48.114828 29998 ts_manager.cc:194] Registered new tserver with Master: ee6ba208af054a77a67f4e0598cd8c63 (127.15.82.67:38555)
I20260929 10:35:48.115403 29998 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.82.67@KRBTEST.COM'} at 127.15.82.67:60831
I20260929 10:35:48.120761 15689 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 10:35:48.124818 30188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:48.121374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47010 (local address 127.15.82.126:37881)
0929 10:35:48.121508 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0929 10:35:48.121526 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0929 10:35:48.121720 (+ 194us) server_negotiation.cc:407] Connection header received
0929 10:35:48.121840 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:35:48.121842 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:35:48.121931 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:35:48.122009 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:35:48.122217 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:48.122781 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:48.123579 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:48.123782 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:48.123851 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:48.124108 (+ 257us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:35:48.124471 (+ 363us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:35:48.124561 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0929 10:35:48.124623 (+ 62us) server_negotiation.cc:299] Negotiation successful
0929 10:35:48.124665 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260929 10:35:48.125440 29998 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:47010:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 10:35:48.129114 29998 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:35:48.137483 30120 tablet_service.cc:1467] Processing CreateTablet for tablet 7e37caac011d47739dae44a915031421 (DEFAULT_TABLE table=test-table [id=b8f3021f8cf74134a7a6e75205b3096c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:35:48.138200 30120 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e37caac011d47739dae44a915031421. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:35:48.141232 30078 tablet_bootstrap.cc:492] T 7e37caac011d47739dae44a915031421 P f379905b6ee84cc6bacf15584dcfd086: Bootstrap starting.
I20260929 10:35:48.142132 30078 tablet_bootstrap.cc:654] T 7e37caac011d47739dae44a915031421 P f379905b6ee84cc6bacf15584dcfd086: Neither blocks nor log segments found. Creating new log.
I20260929 10:35:48.142419 30193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:48.136439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.66:38923 (local address 127.0.0.1:53784)
0929 10:35:48.136949 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:48.136967 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 10:35:48.137042 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:35:48.138035 (+ 993us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:35:48.138041 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:35:48.138065 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:35:48.138183 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:48.138190 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:48.139603 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:48.139606 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:48.140777 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:48.140787 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:48.141848 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:48.141852 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:48.141963 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:48.142017 (+ 54us) client_negotiation.cc:769] Sending connection context
0929 10:35:48.142086 (+ 69us) client_negotiation.cc:240] Negotiation successful
0929 10:35:48.142219 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"mutex_wait_us":9}
I20260929 10:35:48.142433 30467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:48.136655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53784 (local address 127.15.82.66:38923)
0929 10:35:48.137490 (+ 835us) server_negotiation.cc:206] Beginning negotiation
0929 10:35:48.137504 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 10:35:48.137549 (+ 45us) server_negotiation.cc:407] Connection header received
0929 10:35:48.137630 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:35:48.137635 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:35:48.137866 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:35:48.138035 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:35:48.138302 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:48.139457 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:48.140902 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:48.141728 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:48.141841 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:48.142057 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0929 10:35:48.142121 (+ 64us) server_negotiation.cc:299] Negotiation successful
0929 10:35:48.142219 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":730,"thread_start_us":151,"threads_started":1}
I20260929 10:35:48.142897 30466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:48.136662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.67:38555 (local address 127.0.0.1:54422)
0929 10:35:48.137149 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:48.137163 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 10:35:48.137248 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:35:48.138113 (+ 865us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:35:48.138115 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:35:48.138127 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:35:48.138189 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:48.138193 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:48.139454 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:48.139461 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:48.140865 (+ 1404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:48.140873 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:48.142109 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:48.142112 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:48.142305 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:48.142326 (+ 21us) client_negotiation.cc:769] Sending connection context
0929 10:35:48.142389 (+ 63us) client_negotiation.cc:240] Negotiation successful
0929 10:35:48.142767 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":422,"mutex_wait_us":96,"thread_start_us":124,"threads_started":1}
I20260929 10:35:48.143813 30078 tablet_bootstrap.cc:492] T 7e37caac011d47739dae44a915031421 P f379905b6ee84cc6bacf15584dcfd086: No bootstrap required, opened a new log
I20260929 10:35:48.143898 30078 ts_tablet_manager.cc:1397] T 7e37caac011d47739dae44a915031421 P f379905b6ee84cc6bacf15584dcfd086: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260929 10:35:48.144111 30468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:48.137242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54422 (local address 127.15.82.67:38555)
0929 10:35:48.137607 (+ 365us) server_negotiation.cc:206] Beginning negotiation
0929 10:35:48.137618 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 10:35:48.137647 (+ 29us) server_negotiation.cc:407] Connection header received
0929 10:35:48.137742 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:35:48.137748 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:35:48.137941 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:35:48.138342 (+ 401us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:35:48.138528 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:48.139321 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:48.140986 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:48.141972 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:48.143300 (+ 1328us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:48.143605 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0929 10:35:48.143656 (+ 51us) server_negotiation.cc:299] Negotiation successful
0929 10:35:48.143753 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":184,"threads_started":1}
I20260929 10:35:48.144380 30078 raft_consensus.cc:348] T 7e37caac011d47739dae44a915031421 P f379905b6ee84cc6bacf15584dcfd086 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c53d47f2877d47dc9e0e6d6793562c71" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 38923 } } peers { permanent_uuid: "ee6ba208af054a77a67f4e0598cd8c63" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 38555 } } peers { permanent_uuid: "f379905b6ee84cc6bacf15584dcfd086" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34003 } }
I20260929 10:35:48.144501 30078 raft_consensus.cc:374] T 7e37caac011d47739dae44a915031421 P f379905b6ee84cc6bacf15584dcfd086 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:35:48.144539 30078 raft_consensus.cc:729] T 7e37caac011d47739dae44a915031421 P f379905b6ee84cc6bacf15584dcfd086 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f379905b6ee84cc6bacf15584dcfd086, State: Initialized, Role: FOLLOWER
I20260929 10:35:48.144194 30259 tablet_service.cc:1467] Processing CreateTablet for tablet 7e37caac011d47739dae44a915031421 (DEFAULT_TABLE table=test-table [id=b8f3021f8cf74134a7a6e75205b3096c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:35:48.144671 30078 consensus_queue.cc:260] T 7e37caac011d47739dae44a915031421 P f379905b6ee84cc6bacf15584dcfd086 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c53d47f2877d47dc9e0e6d6793562c71" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 38923 } } peers { permanent_uuid: "ee6ba208af054a77a67f4e0598cd8c63" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 38555 } } peers { permanent_uuid: "f379905b6ee84cc6bacf15584dcfd086" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34003 } }
I20260929 10:35:48.144953 30078 ts_tablet_manager.cc:1428] T 7e37caac011d47739dae44a915031421 P f379905b6ee84cc6bacf15584dcfd086: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260929 10:35:48.145314 30259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e37caac011d47739dae44a915031421. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:35:48.149529 30391 tablet_service.cc:1467] Processing CreateTablet for tablet 7e37caac011d47739dae44a915031421 (DEFAULT_TABLE table=test-table [id=b8f3021f8cf74134a7a6e75205b3096c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:35:48.150460 30217 tablet_bootstrap.cc:492] T 7e37caac011d47739dae44a915031421 P c53d47f2877d47dc9e0e6d6793562c71: Bootstrap starting.
I20260929 10:35:48.150636 30391 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e37caac011d47739dae44a915031421. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:35:48.152426 30217 tablet_bootstrap.cc:654] T 7e37caac011d47739dae44a915031421 P c53d47f2877d47dc9e0e6d6793562c71: Neither blocks nor log segments found. Creating new log.
I20260929 10:35:48.153309 30217 log.cc:824] T 7e37caac011d47739dae44a915031421 P c53d47f2877d47dc9e0e6d6793562c71: Log is configured to *not* fsync() on all Append() calls
I20260929 10:35:48.155097 30217 tablet_bootstrap.cc:492] T 7e37caac011d47739dae44a915031421 P c53d47f2877d47dc9e0e6d6793562c71: No bootstrap required, opened a new log
I20260929 10:35:48.155285 30217 ts_tablet_manager.cc:1397] T 7e37caac011d47739dae44a915031421 P c53d47f2877d47dc9e0e6d6793562c71: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260929 10:35:48.157356 30471 tablet_bootstrap.cc:492] T 7e37caac011d47739dae44a915031421 P ee6ba208af054a77a67f4e0598cd8c63: Bootstrap starting.
I20260929 10:35:48.158571 30217 raft_consensus.cc:348] T 7e37caac011d47739dae44a915031421 P c53d47f2877d47dc9e0e6d6793562c71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c53d47f2877d47dc9e0e6d6793562c71" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 38923 } } peers { permanent_uuid: "ee6ba208af054a77a67f4e0598cd8c63" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 38555 } } peers { permanent_uuid: "f379905b6ee84cc6bacf15584dcfd086" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34003 } }
I20260929 10:35:48.158834 30217 raft_consensus.cc:374] T 7e37caac011d47739dae44a915031421 P c53d47f2877d47dc9e0e6d6793562c71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:35:48.158918 30217 raft_consensus.cc:729] T 7e37caac011d47739dae44a915031421 P c53d47f2877d47dc9e0e6d6793562c71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c53d47f2877d47dc9e0e6d6793562c71, State: Initialized, Role: FOLLOWER
I20260929 10:35:48.159341 30217 consensus_queue.cc:260] T 7e37caac011d47739dae44a915031421 P c53d47f2877d47dc9e0e6d6793562c71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c53d47f2877d47dc9e0e6d6793562c71" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 38923 } } peers { permanent_uuid: "ee6ba208af054a77a67f4e0598cd8c63" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 38555 } } peers { permanent_uuid: "f379905b6ee84cc6bacf15584dcfd086" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34003 } }
I20260929 10:35:48.160022 30471 tablet_bootstrap.cc:654] T 7e37caac011d47739dae44a915031421 P ee6ba208af054a77a67f4e0598cd8c63: Neither blocks nor log segments found. Creating new log.
I20260929 10:35:48.160033 30325 heartbeater.cc:503] Master 127.15.82.126:37881 was elected leader, sending a full tablet report...
I20260929 10:35:48.160303 30217 ts_tablet_manager.cc:1428] T 7e37caac011d47739dae44a915031421 P c53d47f2877d47dc9e0e6d6793562c71: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260929 10:35:48.160897 30471 log.cc:824] T 7e37caac011d47739dae44a915031421 P ee6ba208af054a77a67f4e0598cd8c63: Log is configured to *not* fsync() on all Append() calls
W20260929 10:35:48.161875 30187 tablet.cc:2369] T 7e37caac011d47739dae44a915031421 P f379905b6ee84cc6bacf15584dcfd086: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 10:35:48.163264 30471 tablet_bootstrap.cc:492] T 7e37caac011d47739dae44a915031421 P ee6ba208af054a77a67f4e0598cd8c63: No bootstrap required, opened a new log
I20260929 10:35:48.163471 30471 ts_tablet_manager.cc:1397] T 7e37caac011d47739dae44a915031421 P ee6ba208af054a77a67f4e0598cd8c63: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260929 10:35:48.166574 30471 raft_consensus.cc:348] T 7e37caac011d47739dae44a915031421 P ee6ba208af054a77a67f4e0598cd8c63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c53d47f2877d47dc9e0e6d6793562c71" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 38923 } } peers { permanent_uuid: "ee6ba208af054a77a67f4e0598cd8c63" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 38555 } } peers { permanent_uuid: "f379905b6ee84cc6bacf15584dcfd086" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34003 } }
I20260929 10:35:48.166802 30471 raft_consensus.cc:374] T 7e37caac011d47739dae44a915031421 P ee6ba208af054a77a67f4e0598cd8c63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:35:48.166873 30471 raft_consensus.cc:729] T 7e37caac011d47739dae44a915031421 P ee6ba208af054a77a67f4e0598cd8c63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee6ba208af054a77a67f4e0598cd8c63, State: Initialized, Role: FOLLOWER
I20260929 10:35:48.167339 30471 consensus_queue.cc:260] T 7e37caac011d47739dae44a915031421 P ee6ba208af054a77a67f4e0598cd8c63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c53d47f2877d47dc9e0e6d6793562c71" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 38923 } } peers { permanent_uuid: "ee6ba208af054a77a67f4e0598cd8c63" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 38555 } } peers { permanent_uuid: "f379905b6ee84cc6bacf15584dcfd086" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34003 } }
I20260929 10:35:48.168042 30457 heartbeater.cc:503] Master 127.15.82.126:37881 was elected leader, sending a full tablet report...
I20260929 10:35:48.168336 30471 ts_tablet_manager.cc:1428] T 7e37caac011d47739dae44a915031421 P ee6ba208af054a77a67f4e0598cd8c63: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260929 10:35:48.255473 30192 raft_consensus.cc:482] T 7e37caac011d47739dae44a915031421 P f379905b6ee84cc6bacf15584dcfd086 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:35:48.255787 30192 raft_consensus.cc:504] T 7e37caac011d47739dae44a915031421 P f379905b6ee84cc6bacf15584dcfd086 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c53d47f2877d47dc9e0e6d6793562c71" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 38923 } } peers { permanent_uuid: "ee6ba208af054a77a67f4e0598cd8c63" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 38555 } } peers { permanent_uuid: "f379905b6ee84cc6bacf15584dcfd086" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34003 } }
I20260929 10:35:48.257148 30192 leader_election.cc:288] T 7e37caac011d47739dae44a915031421 P f379905b6ee84cc6bacf15584dcfd086 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c53d47f2877d47dc9e0e6d6793562c71 (127.15.82.66:38923), ee6ba208af054a77a67f4e0598cd8c63 (127.15.82.67:38555)
I20260929 10:35:48.262720 30467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:48.257542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.65:42585 (local address 127.15.82.66:38923)
0929 10:35:48.257688 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0929 10:35:48.257693 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:35:48.257881 (+ 188us) server_negotiation.cc:407] Connection header received
0929 10:35:48.258093 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:35:48.258097 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:35:48.258153 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:35:48.258240 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:35:48.258715 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:48.259502 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:48.261330 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:48.261969 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:48.262107 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:48.262166 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0929 10:35:48.262486 (+ 320us) server_negotiation.cc:299] Negotiation successful
0929 10:35:48.262583 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260929 10:35:48.262776 30475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:48.257389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.66:38923 (local address 127.15.82.65:42585)
0929 10:35:48.257760 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:48.257778 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 10:35:48.257932 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:35:48.258284 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:35:48.258287 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:35:48.258309 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:35:48.258558 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:48.258571 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:48.259650 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:48.259654 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:48.261158 (+ 1504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:48.261172 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:48.262140 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:48.262144 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:48.262261 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:48.262330 (+ 69us) client_negotiation.cc:769] Sending connection context
0929 10:35:48.262469 (+ 139us) client_negotiation.cc:240] Negotiation successful
0929 10:35:48.262568 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"mutex_wait_us":93,"thread_start_us":102,"threads_started":1}
I20260929 10:35:48.262981 30468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:48.257607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.65:55509 (local address 127.15.82.67:38555)
0929 10:35:48.257745 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0929 10:35:48.257749 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:35:48.257976 (+ 227us) server_negotiation.cc:407] Connection header received
0929 10:35:48.258093 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:35:48.258097 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:35:48.258152 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:35:48.258240 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:35:48.258715 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:48.259504 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:48.261330 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:48.261969 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:48.262107 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:48.262166 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0929 10:35:48.262689 (+ 523us) server_negotiation.cc:299] Negotiation successful
0929 10:35:48.262772 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260929 10:35:48.263362 30476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:48.257502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.67:38555 (local address 127.15.82.65:55509)
0929 10:35:48.257882 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:48.257896 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 10:35:48.257983 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:35:48.258284 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:35:48.258287 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:35:48.258366 (+ 79us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:35:48.258557 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:48.258571 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:48.259650 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:48.259654 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:48.261159 (+ 1505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:48.261172 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:48.262392 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:48.262437 (+ 45us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:48.262535 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:48.262558 (+ 23us) client_negotiation.cc:769] Sending connection context
0929 10:35:48.263124 (+ 566us) client_negotiation.cc:240] Negotiation successful
0929 10:35:48.263217 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"mutex_wait_us":245,"spinlock_wait_cycles":21760,"thread_start_us":91,"threads_started":1}
I20260929 10:35:48.263567 30279 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e37caac011d47739dae44a915031421" candidate_uuid: "f379905b6ee84cc6bacf15584dcfd086" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c53d47f2877d47dc9e0e6d6793562c71" is_pre_election: true
I20260929 10:35:48.263947 30279 raft_consensus.cc:2454] T 7e37caac011d47739dae44a915031421 P c53d47f2877d47dc9e0e6d6793562c71 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f379905b6ee84cc6bacf15584dcfd086 in term 0.
I20260929 10:35:48.264117 30411 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e37caac011d47739dae44a915031421" candidate_uuid: "f379905b6ee84cc6bacf15584dcfd086" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee6ba208af054a77a67f4e0598cd8c63" is_pre_election: true
I20260929 10:35:48.264467 30411 raft_consensus.cc:2454] T 7e37caac011d47739dae44a915031421 P ee6ba208af054a77a67f4e0598cd8c63 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f379905b6ee84cc6bacf15584dcfd086 in term 0.
I20260929 10:35:48.264500 30075 leader_election.cc:302] T 7e37caac011d47739dae44a915031421 P f379905b6ee84cc6bacf15584dcfd086 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c53d47f2877d47dc9e0e6d6793562c71, f379905b6ee84cc6bacf15584dcfd086; no voters:
I20260929 10:35:48.264833 30192 raft_consensus.cc:2781] T 7e37caac011d47739dae44a915031421 P f379905b6ee84cc6bacf15584dcfd086 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:35:48.264935 30192 raft_consensus.cc:482] T 7e37caac011d47739dae44a915031421 P f379905b6ee84cc6bacf15584dcfd086 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:35:48.264981 30192 raft_consensus.cc:3037] T 7e37caac011d47739dae44a915031421 P f379905b6ee84cc6bacf15584dcfd086 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:35:48.266078 30192 raft_consensus.cc:504] T 7e37caac011d47739dae44a915031421 P f379905b6ee84cc6bacf15584dcfd086 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c53d47f2877d47dc9e0e6d6793562c71" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 38923 } } peers { permanent_uuid: "ee6ba208af054a77a67f4e0598cd8c63" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 38555 } } peers { permanent_uuid: "f379905b6ee84cc6bacf15584dcfd086" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34003 } }
I20260929 10:35:48.266490 30192 leader_election.cc:288] T 7e37caac011d47739dae44a915031421 P f379905b6ee84cc6bacf15584dcfd086 [CANDIDATE]: Term 1 election: Requested vote from peers c53d47f2877d47dc9e0e6d6793562c71 (127.15.82.66:38923), ee6ba208af054a77a67f4e0598cd8c63 (127.15.82.67:38555)
I20260929 10:35:48.266837 30279 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e37caac011d47739dae44a915031421" candidate_uuid: "f379905b6ee84cc6bacf15584dcfd086" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c53d47f2877d47dc9e0e6d6793562c71"
I20260929 10:35:48.266968 30411 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e37caac011d47739dae44a915031421" candidate_uuid: "f379905b6ee84cc6bacf15584dcfd086" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee6ba208af054a77a67f4e0598cd8c63"
I20260929 10:35:48.266983 30279 raft_consensus.cc:3037] T 7e37caac011d47739dae44a915031421 P c53d47f2877d47dc9e0e6d6793562c71 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:35:48.267122 30411 raft_consensus.cc:3037] T 7e37caac011d47739dae44a915031421 P ee6ba208af054a77a67f4e0598cd8c63 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:35:48.268096 30279 raft_consensus.cc:2454] T 7e37caac011d47739dae44a915031421 P c53d47f2877d47dc9e0e6d6793562c71 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f379905b6ee84cc6bacf15584dcfd086 in term 1.
I20260929 10:35:48.268272 30411 raft_consensus.cc:2454] T 7e37caac011d47739dae44a915031421 P ee6ba208af054a77a67f4e0598cd8c63 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f379905b6ee84cc6bacf15584dcfd086 in term 1.
I20260929 10:35:48.268427 30075 leader_election.cc:302] T 7e37caac011d47739dae44a915031421 P f379905b6ee84cc6bacf15584dcfd086 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c53d47f2877d47dc9e0e6d6793562c71, f379905b6ee84cc6bacf15584dcfd086; no voters:
I20260929 10:35:48.268622 30192 raft_consensus.cc:2781] T 7e37caac011d47739dae44a915031421 P f379905b6ee84cc6bacf15584dcfd086 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:35:48.268821 30192 raft_consensus.cc:686] T 7e37caac011d47739dae44a915031421 P f379905b6ee84cc6bacf15584dcfd086 [term 1 LEADER]: Becoming Leader. State: Replica: f379905b6ee84cc6bacf15584dcfd086, State: Running, Role: LEADER
I20260929 10:35:48.269104 30192 consensus_queue.cc:237] T 7e37caac011d47739dae44a915031421 P f379905b6ee84cc6bacf15584dcfd086 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c53d47f2877d47dc9e0e6d6793562c71" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 38923 } } peers { permanent_uuid: "ee6ba208af054a77a67f4e0598cd8c63" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 38555 } } peers { permanent_uuid: "f379905b6ee84cc6bacf15584dcfd086" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34003 } }
I20260929 10:35:48.272171 29997 catalog_manager.cc:5495] T 7e37caac011d47739dae44a915031421 P f379905b6ee84cc6bacf15584dcfd086 reported cstate change: term changed from 0 to 1, leader changed from <none> to f379905b6ee84cc6bacf15584dcfd086 (127.15.82.65). New cstate: current_term: 1 leader_uuid: "f379905b6ee84cc6bacf15584dcfd086" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c53d47f2877d47dc9e0e6d6793562c71" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 38923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee6ba208af054a77a67f4e0598cd8c63" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 38555 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f379905b6ee84cc6bacf15584dcfd086" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34003 } health_report { overall_health: HEALTHY } } }
I20260929 10:35:48.302055 30479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:48.297904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46060 (local address 127.15.82.65:34003)
0929 10:35:48.298277 (+ 373us) server_negotiation.cc:206] Beginning negotiation
0929 10:35:48.298284 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 10:35:48.298298 (+ 14us) server_negotiation.cc:407] Connection header received
0929 10:35:48.298375 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:35:48.298377 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:35:48.298495 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:35:48.298640 (+ 145us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:35:48.298891 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:48.299451 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:48.300370 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:48.300620 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:48.300732 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:48.301066 (+ 334us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:35:48.301530 (+ 464us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:35:48.301671 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0929 10:35:48.301784 (+ 113us) server_negotiation.cc:299] Negotiation successful
0929 10:35:48.301843 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":159,"threads_started":1}
I20260929 10:35:48.306752 30411 raft_consensus.cc:1261] T 7e37caac011d47739dae44a915031421 P ee6ba208af054a77a67f4e0598cd8c63 [term 1 FOLLOWER]: Refusing update from remote peer f379905b6ee84cc6bacf15584dcfd086: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:35:48.306751 30279 raft_consensus.cc:1261] T 7e37caac011d47739dae44a915031421 P c53d47f2877d47dc9e0e6d6793562c71 [term 1 FOLLOWER]: Refusing update from remote peer f379905b6ee84cc6bacf15584dcfd086: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:35:48.307610 30477 consensus_queue.cc:1035] T 7e37caac011d47739dae44a915031421 P f379905b6ee84cc6bacf15584dcfd086 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee6ba208af054a77a67f4e0598cd8c63" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 38555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:35:48.307808 30192 consensus_queue.cc:1035] T 7e37caac011d47739dae44a915031421 P f379905b6ee84cc6bacf15584dcfd086 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c53d47f2877d47dc9e0e6d6793562c71" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 38923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:35:48.324968 29997 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:47010:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:35:48.325152 29997 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:47010:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:35:48.327540 29997 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 00ca51f69920468e9052d7591f5043ec: Sending DeleteTablet for 3 replicas of tablet 7e37caac011d47739dae44a915031421
I20260929 10:35:48.328259 30259 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e37caac011d47739dae44a915031421 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:35:48 UTC) from {username='kudu', principal='kudu/127.15.82.126@KRBTEST.COM'} at 127.0.0.1:53784
I20260929 10:35:48.328418 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 30036
I20260929 10:35:48.328541 30120 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e37caac011d47739dae44a915031421 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:35:48 UTC) from {username='kudu', principal='kudu/127.15.82.126@KRBTEST.COM'} at 127.0.0.1:46048
I20260929 10:35:48.328760 30391 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e37caac011d47739dae44a915031421 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:35:48 UTC) from {username='kudu', principal='kudu/127.15.82.126@KRBTEST.COM'} at 127.0.0.1:54422
I20260929 10:35:48.329219 30491 tablet_replica.cc:330] T 7e37caac011d47739dae44a915031421 P ee6ba208af054a77a67f4e0598cd8c63: stopping tablet replica
I20260929 10:35:48.329566 30491 raft_consensus.cc:2229] T 7e37caac011d47739dae44a915031421 P ee6ba208af054a77a67f4e0598cd8c63 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:35:48.329885 30491 raft_consensus.cc:2258] T 7e37caac011d47739dae44a915031421 P ee6ba208af054a77a67f4e0598cd8c63 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:35:48.330343 30490 tablet_replica.cc:330] T 7e37caac011d47739dae44a915031421 P c53d47f2877d47dc9e0e6d6793562c71: stopping tablet replica
I20260929 10:35:48.330690 30490 raft_consensus.cc:2229] T 7e37caac011d47739dae44a915031421 P c53d47f2877d47dc9e0e6d6793562c71 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:35:48.331002 30490 raft_consensus.cc:2258] T 7e37caac011d47739dae44a915031421 P c53d47f2877d47dc9e0e6d6793562c71 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:35:48.331699 30491 ts_tablet_manager.cc:1905] T 7e37caac011d47739dae44a915031421 P ee6ba208af054a77a67f4e0598cd8c63: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:35:48.332255 30490 ts_tablet_manager.cc:1905] T 7e37caac011d47739dae44a915031421 P c53d47f2877d47dc9e0e6d6793562c71: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:35:48.334834 30490 ts_tablet_manager.cc:1918] T 7e37caac011d47739dae44a915031421 P c53d47f2877d47dc9e0e6d6793562c71: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:35:48.334839 30491 ts_tablet_manager.cc:1918] T 7e37caac011d47739dae44a915031421 P ee6ba208af054a77a67f4e0598cd8c63: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:35:48.334923 30490 log.cc:1196] T 7e37caac011d47739dae44a915031421 P c53d47f2877d47dc9e0e6d6793562c71: Deleting WAL directory at /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-1/wal/wals/7e37caac011d47739dae44a915031421
I20260929 10:35:48.334944 30491 log.cc:1196] T 7e37caac011d47739dae44a915031421 P ee6ba208af054a77a67f4e0598cd8c63: Deleting WAL directory at /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-2/wal/wals/7e37caac011d47739dae44a915031421
I20260929 10:35:48.335223 30490 ts_tablet_manager.cc:1939] T 7e37caac011d47739dae44a915031421 P c53d47f2877d47dc9e0e6d6793562c71: Deleting consensus metadata
I20260929 10:35:48.335290 30491 ts_tablet_manager.cc:1939] T 7e37caac011d47739dae44a915031421 P ee6ba208af054a77a67f4e0598cd8c63: Deleting consensus metadata
W20260929 10:35:48.336139 29982 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.82.65:34003 (error 108)
I20260929 10:35:48.336237 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 30191
I20260929 10:35:48.337714 29983 catalog_manager.cc:4841] TS ee6ba208af054a77a67f4e0598cd8c63 (127.15.82.67:38555): tablet 7e37caac011d47739dae44a915031421 (table test-table [id=b8f3021f8cf74134a7a6e75205b3096c]) successfully deleted
I20260929 10:35:48.338016 30466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:48.337663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.65:34003 (local address 127.0.0.1:46066)
0929 10:35:48.337818 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:48.337900 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.65:34003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260929 10:35:48.338224 29982 catalog_manager.cc:4584] TS f379905b6ee84cc6bacf15584dcfd086 (127.15.82.65:34003): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7e37caac011d47739dae44a915031421: Network error: Client connection negotiation failed: client connection to 127.15.82.65:34003: connect: Connection refused (error 111)
I20260929 10:35:48.342711 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 30328
I20260929 10:35:48.343722 30466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:48.342842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.66:38923 (local address 127.0.0.1:53800)
0929 10:35:48.343603 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:48.343649 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.66:38923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":689}
I20260929 10:35:48.348665 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 29965
I20260929 10:35:48.355594 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:37881
--webserver_interface=127.15.82.126
--webserver_port=45403
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab
--principal=kudu/127.15.82.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:44075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.82.126:37881
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:35:48.473256 30493 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:35:48.473573 30493 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:35:48.473615 30493 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:35:48.477793 30493 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:35:48.477878 30493 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:35:48.477905 30493 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:35:48.477922 30493 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:35:48.477938 30493 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:35:48.483642 30493 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:44075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.82.126:37881
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab
--principal=kudu/127.15.82.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:37881
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.82.126
--webserver_port=45403
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30493
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:35:48.484803 30493 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:35:48.485868 30493 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:35:48.494961 30498 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:35:48.494961 30501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:35:48.494958 30499 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:35:48.495342 30493 server_base.cc:1034] running on GCE node
I20260929 10:35:48.495939 30493 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:35:48.497203 30493 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:35:48.498371 30493 hybrid_clock.cc:648] HybridClock initialized: now 1790678148498348 us; error 41 us; skew 500 ppm
Sep 29 10:35:48 dist-test-slave-vg89 krb5kdc[29416](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678148, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:35:48.501449 30493 init.cc:377] Logged in from keytab as kudu/127.15.82.126@KRBTEST.COM (short username kudu)
I20260929 10:35:48.502549 30493 webserver.cc:460] Webserver started at http://127.15.82.126:45403/ using document root <none> and password file <none>
I20260929 10:35:48.503187 30493 fs_manager.cc:359] Metadata directory not provided
I20260929 10:35:48.503255 30493 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:35:48.507459 30493 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260929 10:35:48.509832 30508 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:35:48.511335 30493 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260929 10:35:48.511497 30493 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/master-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/master-0/wal
uuid: "00ca51f69920468e9052d7591f5043ec"
format_stamp: "Formatted at 2026-09-29 10:35:45 on dist-test-slave-vg89"
server_key: "70936bd37ac113f4769fa6afe351f761"
server_key_iv: "a921de9c6f3d8ac2d245aab071461444"
server_key_version: "encryptionkey@0"
I20260929 10:35:48.512117 30493 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:35:48.526602 30493 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:35:48.529821 30493 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:35:48.530082 30493 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:35:48.537648 30493 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.126:37881
I20260929 10:35:48.538051 30560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.126:37881 every 8 connection(s)
I20260929 10:35:48.538990 30493 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
I20260929 10:35:48.542883 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 30493
I20260929 10:35:48.543392 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.65:34003
--local_ip_for_outbound_sockets=127.15.82.65
--tserver_master_addrs=127.15.82.126:37881
--webserver_port=35431
--webserver_interface=127.15.82.65
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab
--principal=kudu/127.15.82.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:44075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 10:35:48.543881 30561 sys_catalog.cc:263] Verifying existing consensus state
I20260929 10:35:48.546231 30561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00ca51f69920468e9052d7591f5043ec: Bootstrap starting.
I20260929 10:35:48.564278 30561 log.cc:824] T 00000000000000000000000000000000 P 00ca51f69920468e9052d7591f5043ec: Log is configured to *not* fsync() on all Append() calls
I20260929 10:35:48.575805 30561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00ca51f69920468e9052d7591f5043ec: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 10:35:48.576232 30561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00ca51f69920468e9052d7591f5043ec: Bootstrap complete.
I20260929 10:35:48.580611 30561 raft_consensus.cc:348] T 00000000000000000000000000000000 P 00ca51f69920468e9052d7591f5043ec [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ca51f69920468e9052d7591f5043ec" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 37881 } }
I20260929 10:35:48.581037 30561 raft_consensus.cc:729] T 00000000000000000000000000000000 P 00ca51f69920468e9052d7591f5043ec [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00ca51f69920468e9052d7591f5043ec, State: Initialized, Role: FOLLOWER
I20260929 10:35:48.581616 30561 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00ca51f69920468e9052d7591f5043ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ca51f69920468e9052d7591f5043ec" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 37881 } }
I20260929 10:35:48.581743 30561 raft_consensus.cc:388] T 00000000000000000000000000000000 P 00ca51f69920468e9052d7591f5043ec [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:35:48.581816 30561 raft_consensus.cc:482] T 00000000000000000000000000000000 P 00ca51f69920468e9052d7591f5043ec [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:35:48.581938 30561 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 00ca51f69920468e9052d7591f5043ec [term 2 FOLLOWER]: Advancing to term 3
I20260929 10:35:48.584028 30561 raft_consensus.cc:504] T 00000000000000000000000000000000 P 00ca51f69920468e9052d7591f5043ec [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ca51f69920468e9052d7591f5043ec" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 37881 } }
I20260929 10:35:48.584376 30561 leader_election.cc:302] T 00000000000000000000000000000000 P 00ca51f69920468e9052d7591f5043ec [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 00ca51f69920468e9052d7591f5043ec; no voters:
I20260929 10:35:48.584733 30561 leader_election.cc:288] T 00000000000000000000000000000000 P 00ca51f69920468e9052d7591f5043ec [CANDIDATE]: Term 3 election: Requested vote from peers
I20260929 10:35:48.584822 30565 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 00ca51f69920468e9052d7591f5043ec [term 3 FOLLOWER]: Leader election won for term 3
I20260929 10:35:48.585031 30565 raft_consensus.cc:686] T 00000000000000000000000000000000 P 00ca51f69920468e9052d7591f5043ec [term 3 LEADER]: Becoming Leader. State: Replica: 00ca51f69920468e9052d7591f5043ec, State: Running, Role: LEADER
I20260929 10:35:48.585343 30565 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00ca51f69920468e9052d7591f5043ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ca51f69920468e9052d7591f5043ec" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 37881 } }
I20260929 10:35:48.586036 30561 sys_catalog.cc:564] T 00000000000000000000000000000000 P 00ca51f69920468e9052d7591f5043ec [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:35:48.586338 30567 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00ca51f69920468e9052d7591f5043ec [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00ca51f69920468e9052d7591f5043ec. Latest consensus state: current_term: 3 leader_uuid: "00ca51f69920468e9052d7591f5043ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ca51f69920468e9052d7591f5043ec" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 37881 } } }
I20260929 10:35:48.586499 30567 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00ca51f69920468e9052d7591f5043ec [sys.catalog]: This master's current role is: LEADER
I20260929 10:35:48.587136 30566 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00ca51f69920468e9052d7591f5043ec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "00ca51f69920468e9052d7591f5043ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ca51f69920468e9052d7591f5043ec" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 37881 } } }
I20260929 10:35:48.587219 30566 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00ca51f69920468e9052d7591f5043ec [sys.catalog]: This master's current role is: LEADER
I20260929 10:35:48.587934 30569 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:35:48.593586 30569 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:35:48.594242 30569 catalog_manager.cc:1259] Loaded cluster ID: d070de294db6467bb3d6ee46974440f3
I20260929 10:35:48.594303 30569 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260929 10:35:48.595306 30569 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fe4b790f9ad google::LogMessage::Fail() at ??:0
@ 0x7fe4b791288a google::LogMessage::SendToLog() at ??:0
@ 0x7fe4b790f4de google::LogMessage::Flush() at ??:0
@ 0x7fe4b7910a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fe4c224efdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7fe4c224fd2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fe4c224c579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7fe4c2290640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fe4b9b613d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fe4b9b61c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7fe4b9b633f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fe4b9b52512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fe4bb1966db start_thread at ??:0
@ 0x7fe4b640d61f clone at ??:0
W20260929 10:35:48.678010 30563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:35:48.678324 30563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:35:48.678453 30563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:35:48.682687 30563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:35:48.682770 30563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:35:48.682888 30563 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.65
I20260929 10:35:48.688395 30563 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:44075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab
--principal=kudu/127.15.82.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.65:34003
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.82.65
--webserver_port=35431
--tserver_master_addrs=127.15.82.126:37881
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30563
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.65
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:35:48.689576 30563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:35:48.690743 30563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 10:35:48.701246 30563 server_base.cc:1034] running on GCE node
W20260929 10:35:48.701208 30602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:35:48.701447 30603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:35:48.701196 30605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:35:48.702014 30563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:35:48.702771 30563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:35:48.703950 30563 hybrid_clock.cc:648] HybridClock initialized: now 1790678148703913 us; error 46 us; skew 500 ppm
Sep 29 10:35:48 dist-test-slave-vg89 krb5kdc[29416](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678148, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:35:48.706777 30563 init.cc:377] Logged in from keytab as kudu/127.15.82.65@KRBTEST.COM (short username kudu)
I20260929 10:35:48.707715 30563 webserver.cc:460] Webserver started at http://127.15.82.65:35431/ using document root <none> and password file <none>
I20260929 10:35:48.708290 30563 fs_manager.cc:359] Metadata directory not provided
I20260929 10:35:48.708351 30563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:35:48.712460 30563 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260929 10:35:48.714784 30612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:35:48.716140 30563 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260929 10:35:48.716274 30563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-0/wal
uuid: "f379905b6ee84cc6bacf15584dcfd086"
format_stamp: "Formatted at 2026-09-29 10:35:46 on dist-test-slave-vg89"
server_key: "412b7efc5455c0d3d9eb9b84117cdc25"
server_key_iv: "3d151e1d30376524fe048ce203a8084e"
server_key_version: "encryptionkey@0"
I20260929 10:35:48.716840 30563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:35:48.743865 30563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:35:48.747248 30563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:35:48.747458 30563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:35:48.748080 30563 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:35:48.749300 30619 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260929 10:35:48.754932 30563 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260929 10:35:48.755012 30563 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260929 10:35:48.755097 30563 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260929 10:35:48.756748 30563 ts_tablet_manager.cc:610] Registered 1 tablets
I20260929 10:35:48.756819 30563 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260929 10:35:48.757130 30619 tablet_bootstrap.cc:492] T 7e37caac011d47739dae44a915031421 P f379905b6ee84cc6bacf15584dcfd086: Bootstrap starting.
I20260929 10:35:48.767328 30563 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.65:34003
I20260929 10:35:48.767603 30726 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.65:34003 every 8 connection(s)
I20260929 10:35:48.768499 30563 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
I20260929 10:35:48.771307 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 30563
I20260929 10:35:48.771780 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.66:38923
--local_ip_for_outbound_sockets=127.15.82.66
--tserver_master_addrs=127.15.82.126:37881
--webserver_port=40525
--webserver_interface=127.15.82.66
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab
--principal=kudu/127.15.82.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:44075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 10:35:48.777441 30619 log.cc:824] T 7e37caac011d47739dae44a915031421 P f379905b6ee84cc6bacf15584dcfd086: Log is configured to *not* fsync() on all Append() calls
I20260929 10:35:48.788152 30619 tablet_bootstrap.cc:492] T 7e37caac011d47739dae44a915031421 P f379905b6ee84cc6bacf15584dcfd086: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 10:35:48.788614 30619 tablet_bootstrap.cc:492] T 7e37caac011d47739dae44a915031421 P f379905b6ee84cc6bacf15584dcfd086: Bootstrap complete.
I20260929 10:35:48.790055 30619 ts_tablet_manager.cc:1397] T 7e37caac011d47739dae44a915031421 P f379905b6ee84cc6bacf15584dcfd086: Time spent bootstrapping tablet: real 0.033s user 0.028s sys 0.001s
I20260929 10:35:48.793380 30619 raft_consensus.cc:348] T 7e37caac011d47739dae44a915031421 P f379905b6ee84cc6bacf15584dcfd086 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c53d47f2877d47dc9e0e6d6793562c71" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 38923 } } peers { permanent_uuid: "ee6ba208af054a77a67f4e0598cd8c63" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 38555 } } peers { permanent_uuid: "f379905b6ee84cc6bacf15584dcfd086" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34003 } }
I20260929 10:35:48.793938 30619 raft_consensus.cc:729] T 7e37caac011d47739dae44a915031421 P f379905b6ee84cc6bacf15584dcfd086 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f379905b6ee84cc6bacf15584dcfd086, State: Initialized, Role: FOLLOWER
I20260929 10:35:48.794569 30619 consensus_queue.cc:260] T 7e37caac011d47739dae44a915031421 P f379905b6ee84cc6bacf15584dcfd086 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c53d47f2877d47dc9e0e6d6793562c71" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 38923 } } peers { permanent_uuid: "ee6ba208af054a77a67f4e0598cd8c63" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 38555 } } peers { permanent_uuid: "f379905b6ee84cc6bacf15584dcfd086" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34003 } }
I20260929 10:35:48.795722 30619 ts_tablet_manager.cc:1428] T 7e37caac011d47739dae44a915031421 P f379905b6ee84cc6bacf15584dcfd086: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
W20260929 10:35:48.821648 30729 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:35:48.770827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:37881 (local address 127.15.82.65:40687)
0929 10:35:48.772116 (+ 1289us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:48.772180 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0929 10:35:48.773277 (+ 1097us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:35:48.820762 (+ 47485us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:37881: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":1025,"thread_start_us":149,"threads_started":1}
W20260929 10:35:48.822239 30727 heartbeater.cc:644] Failed to heartbeat to 127.15.82.126:37881 (0 consecutive failures): Network error: Failed to ping master at 127.15.82.126:37881: Client connection negotiation failed: client connection to 127.15.82.126:37881: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260929 10:35:48.823014 30729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:48.822748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:37881 (local address 127.15.82.65:56469)
0929 10:35:48.822877 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:48.822961 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260929 10:35:48.823781 30729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:48.823569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:37881 (local address 127.15.82.65:37531)
0929 10:35:48.823681 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:48.823733 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260929 10:35:48.823984 30727 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 10:35:48.896149 30730 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:35:48.896453 30730 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:35:48.896530 30730 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:35:48.901003 30730 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:35:48.901101 30730 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:35:48.901202 30730 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.66
I20260929 10:35:48.906719 30730 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:44075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab
--principal=kudu/127.15.82.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.66:38923
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.82.66
--webserver_port=40525
--tserver_master_addrs=127.15.82.126:37881
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30730
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.66
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:35:48.907846 30730 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:35:48.908931 30730 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:35:48.918731 30740 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:35:48.918818 30730 server_base.cc:1034] running on GCE node
W20260929 10:35:48.918744 30737 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:35:48.918748 30738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:35:48.919375 30730 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:35:48.920104 30730 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:35:48.921258 30730 hybrid_clock.cc:648] HybridClock initialized: now 1790678148921240 us; error 30 us; skew 500 ppm
Sep 29 10:35:48 dist-test-slave-vg89 krb5kdc[29416](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678148, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:35:48.924175 30730 init.cc:377] Logged in from keytab as kudu/127.15.82.66@KRBTEST.COM (short username kudu)
I20260929 10:35:48.925067 30730 webserver.cc:460] Webserver started at http://127.15.82.66:40525/ using document root <none> and password file <none>
I20260929 10:35:48.925648 30730 fs_manager.cc:359] Metadata directory not provided
I20260929 10:35:48.925740 30730 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:35:48.929581 30730 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260929 10:35:48.931687 30747 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:35:48.933166 30730 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 10:35:48.933317 30730 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-1/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-1/wal
uuid: "c53d47f2877d47dc9e0e6d6793562c71"
format_stamp: "Formatted at 2026-09-29 10:35:46 on dist-test-slave-vg89"
server_key: "fa1f3b5e7b477f15c03b5a1d21684acc"
server_key_iv: "0ca6a0f33ea9194c85e94ea0e0e92a63"
server_key_version: "encryptionkey@0"
I20260929 10:35:48.933897 30730 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:35:48.948262 30730 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:35:48.951766 30730 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:35:48.951979 30730 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:35:48.952634 30730 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:35:48.953503 30730 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:35:48.953572 30730 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:35:48.953645 30730 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:35:48.953696 30730 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:35:48.963323 30730 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.66:38923
I20260929 10:35:48.963342 30860 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.66:38923 every 8 connection(s)
I20260929 10:35:48.964360 30730 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
I20260929 10:35:48.968135 30863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:48.966493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:37881 (local address 127.15.82.66:55639)
0929 10:35:48.966985 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:48.967415 (+ 430us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":132,"threads_started":1}
W20260929 10:35:48.968665 30861 heartbeater.cc:644] Failed to heartbeat to 127.15.82.126:37881 (0 consecutive failures): Network error: Failed to ping master at 127.15.82.126:37881: Client connection negotiation failed: client connection to 127.15.82.126:37881: connect: Connection refused (error 111)
I20260929 10:35:48.969273 30863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:48.969047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:37881 (local address 127.15.82.66:51345)
0929 10:35:48.969152 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:48.969202 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260929 10:35:48.970016 30863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:48.969703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:37881 (local address 127.15.82.66:45999)
0929 10:35:48.969883 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:48.969938 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260929 10:35:48.970172 30861 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260929 10:35:48.973142 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 30730
I20260929 10:35:48.973495 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.67:38555
--local_ip_for_outbound_sockets=127.15.82.67
--tserver_master_addrs=127.15.82.126:37881
--webserver_port=37299
--webserver_interface=127.15.82.67
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab
--principal=kudu/127.15.82.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:44075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:35:49.085690 30864 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:35:49.085975 30864 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:35:49.086020 30864 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:35:49.090044 30864 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:35:49.090104 30864 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:35:49.090163 30864 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.67
I20260929 10:35:49.095543 30864 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:44075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab
--principal=kudu/127.15.82.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.67:38555
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.82.67
--webserver_port=37299
--tserver_master_addrs=127.15.82.126:37881
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30864
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.67
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:35:49.096642 30864 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:35:49.097633 30864 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:35:49.107352 30869 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:35:49.107481 30864 server_base.cc:1034] running on GCE node
W20260929 10:35:49.107352 30872 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:35:49.107360 30870 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:35:49.108089 30864 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:35:49.108793 30864 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:35:49.109989 30864 hybrid_clock.cc:648] HybridClock initialized: now 1790678149109946 us; error 52 us; skew 500 ppm
Sep 29 10:35:49 dist-test-slave-vg89 krb5kdc[29416](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678149, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:35:49.112951 30864 init.cc:377] Logged in from keytab as kudu/127.15.82.67@KRBTEST.COM (short username kudu)
I20260929 10:35:49.113835 30864 webserver.cc:460] Webserver started at http://127.15.82.67:37299/ using document root <none> and password file <none>
I20260929 10:35:49.114378 30864 fs_manager.cc:359] Metadata directory not provided
I20260929 10:35:49.114518 30864 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:35:49.118317 30864 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260929 10:35:49.120545 30879 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:35:49.121927 30864 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 10:35:49.122030 30864 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-2/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-2/wal
uuid: "ee6ba208af054a77a67f4e0598cd8c63"
format_stamp: "Formatted at 2026-09-29 10:35:46 on dist-test-slave-vg89"
server_key: "3ecac6d030922b70a081b8bd106edd1c"
server_key_iv: "49980e6545024fe31ced90f03bc88c82"
server_key_version: "encryptionkey@0"
I20260929 10:35:49.122637 30864 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:35:49.143718 30864 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:35:49.146962 30864 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:35:49.147131 30864 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:35:49.147696 30864 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:35:49.148484 30864 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:35:49.148541 30864 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:35:49.148582 30864 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:35:49.148598 30864 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:35:49.157567 30864 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.67:38555
I20260929 10:35:49.157586 30992 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.67:38555 every 8 connection(s)
I20260929 10:35:49.158701 30864 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
I20260929 10:35:49.160351 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 30864
I20260929 10:35:49.162662 30996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:49.160969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:37881 (local address 127.15.82.67:35279)
0929 10:35:49.161472 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:49.161620 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":123,"threads_started":1}
W20260929 10:35:49.163100 30993 heartbeater.cc:644] Failed to heartbeat to 127.15.82.126:37881 (0 consecutive failures): Network error: Failed to ping master at 127.15.82.126:37881: Client connection negotiation failed: client connection to 127.15.82.126:37881: connect: Connection refused (error 111)
I20260929 10:35:49.163703 30996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:49.163488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:37881 (local address 127.15.82.67:46159)
0929 10:35:49.163603 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:49.163653 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260929 10:35:49.164305 30996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:49.164108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:37881 (local address 127.15.82.67:38587)
0929 10:35:49.164211 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:49.164254 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260929 10:35:49.164528 30993 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260929 10:35:49.824998 30997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:49.824568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:37881 (local address 127.15.82.65:52367)
0929 10:35:49.824828 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:49.824886 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":97,"threads_started":1}
I20260929 10:35:49.971218 30998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:49.970791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:37881 (local address 127.15.82.66:41061)
0929 10:35:49.971047 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:49.971106 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":70,"threads_started":1}
I20260929 10:35:50.165534 30999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:50.165085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:37881 (local address 127.15.82.67:55353)
0929 10:35:50.165354 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:50.165424 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":69,"threads_started":1}
I20260929 10:35:50.602690 31000 raft_consensus.cc:482] T 7e37caac011d47739dae44a915031421 P f379905b6ee84cc6bacf15584dcfd086 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:35:50.603035 31000 raft_consensus.cc:504] T 7e37caac011d47739dae44a915031421 P f379905b6ee84cc6bacf15584dcfd086 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c53d47f2877d47dc9e0e6d6793562c71" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 38923 } } peers { permanent_uuid: "ee6ba208af054a77a67f4e0598cd8c63" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 38555 } } peers { permanent_uuid: "f379905b6ee84cc6bacf15584dcfd086" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34003 } }
I20260929 10:35:50.604259 31000 leader_election.cc:288] T 7e37caac011d47739dae44a915031421 P f379905b6ee84cc6bacf15584dcfd086 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c53d47f2877d47dc9e0e6d6793562c71 (127.15.82.66:38923), ee6ba208af054a77a67f4e0598cd8c63 (127.15.82.67:38555)
Sep 29 10:35:50 dist-test-slave-vg89 krb5kdc[29416](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678148, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.65@KRBTEST.COM for kudu/127.15.82.67@KRBTEST.COM
Sep 29 10:35:50 dist-test-slave-vg89 krb5kdc[29416](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678148, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.65@KRBTEST.COM for kudu/127.15.82.66@KRBTEST.COM
I20260929 10:35:50.621609 31003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:50.604672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.65:59843 (local address 127.15.82.67:38555)
0929 10:35:50.605127 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0929 10:35:50.605149 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0929 10:35:50.605193 (+ 44us) server_negotiation.cc:407] Connection header received
0929 10:35:50.606143 (+ 950us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:35:50.606152 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:35:50.606671 (+ 519us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:35:50.607063 (+ 392us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:35:50.609003 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:50.610622 (+ 1619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:50.612639 (+ 2017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:50.613404 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:50.613601 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:50.616611 (+ 3010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:35:50.616616 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:35:50.616629 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:35:50.616670 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:35:50.618761 (+ 2091us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:35:50.619317 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:35:50.619322 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:35:50.619326 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:35:50.619440 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:35:50.619862 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:35:50.619866 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:35:50.619868 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:35:50.620400 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:35:50.620557 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0929 10:35:50.621141 (+ 584us) server_negotiation.cc:299] Negotiation successful
0929 10:35:50.621399 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":95,"threads_started":1}
I20260929 10:35:50.621596 31001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:50.604471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.67:38555 (local address 127.15.82.65:59843)
0929 10:35:50.604849 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:50.604859 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 10:35:50.604916 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:35:50.607177 (+ 2261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:35:50.607184 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:35:50.607819 (+ 635us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:35:50.608755 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:50.608778 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:50.610779 (+ 2001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:50.610782 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:50.612489 (+ 1707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:50.612501 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:50.613545 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:50.613548 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:50.613775 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:50.614686 (+ 911us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:35:50.614697 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:35:50.616328 (+ 1631us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:35:50.618892 (+ 2564us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:35:50.618898 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:35:50.618913 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:35:50.619189 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:35:50.619566 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:35:50.619569 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:35:50.619571 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:35:50.619774 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:35:50.620535 (+ 761us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:35:50.620541 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:35:50.620786 (+ 245us) client_negotiation.cc:769] Sending connection context
0929 10:35:50.621016 (+ 230us) client_negotiation.cc:240] Negotiation successful
0929 10:35:50.621410 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"mutex_wait_us":411,"spinlock_wait_cycles":7424,"thread_start_us":70,"threads_started":1}
I20260929 10:35:50.621739 31002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:50.604471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.66:38923 (local address 127.15.82.65:53185)
0929 10:35:50.604774 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:50.604788 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 10:35:50.604906 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:35:50.607177 (+ 2271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:35:50.607184 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:35:50.607819 (+ 635us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:35:50.608755 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:50.608778 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:50.610779 (+ 2001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:50.610782 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:50.612489 (+ 1707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:35:50.612507 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:35:50.613545 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:35:50.613548 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:35:50.613911 (+ 363us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:50.614635 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:35:50.614666 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:35:50.617191 (+ 2525us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:35:50.619451 (+ 2260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:35:50.619454 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:35:50.619456 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:35:50.619669 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:35:50.620062 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:35:50.620065 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:35:50.620067 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:35:50.620181 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:35:50.620833 (+ 652us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:35:50.620836 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:35:50.620938 (+ 102us) client_negotiation.cc:769] Sending connection context
0929 10:35:50.621083 (+ 145us) client_negotiation.cc:240] Negotiation successful
0929 10:35:50.621505 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"mutex_wait_us":721,"spinlock_wait_cycles":8320,"thread_start_us":63,"threads_started":1}
I20260929 10:35:50.622102 31004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:50.604682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.65:53185 (local address 127.15.82.66:38923)
0929 10:35:50.605127 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0929 10:35:50.605144 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0929 10:35:50.605193 (+ 49us) server_negotiation.cc:407] Connection header received
0929 10:35:50.606143 (+ 950us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:35:50.606152 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:35:50.606671 (+ 519us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:35:50.607063 (+ 392us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:35:50.609003 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:50.610622 (+ 1619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:50.612639 (+ 2017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:35:50.613406 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:35:50.613602 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:35:50.617420 (+ 3818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:35:50.617425 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:35:50.617437 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:35:50.617465 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:35:50.619310 (+ 1845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:35:50.619792 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:35:50.619800 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:35:50.619810 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:35:50.619953 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:35:50.620309 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:35:50.620313 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:35:50.620315 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:35:50.620717 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:35:50.620851 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0929 10:35:50.621285 (+ 434us) server_negotiation.cc:299] Negotiation successful
0929 10:35:50.621589 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":119,"threads_started":1}
W20260929 10:35:50.624548 30616 leader_election.cc:341] T 7e37caac011d47739dae44a915031421 P f379905b6ee84cc6bacf15584dcfd086 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c53d47f2877d47dc9e0e6d6793562c71 (127.15.82.66:38923): Not found: Tablet not found: 7e37caac011d47739dae44a915031421
W20260929 10:35:50.624742 30614 leader_election.cc:341] T 7e37caac011d47739dae44a915031421 P f379905b6ee84cc6bacf15584dcfd086 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ee6ba208af054a77a67f4e0598cd8c63 (127.15.82.67:38555): Not found: Tablet not found: 7e37caac011d47739dae44a915031421
I20260929 10:35:50.624846 30614 leader_election.cc:302] T 7e37caac011d47739dae44a915031421 P f379905b6ee84cc6bacf15584dcfd086 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f379905b6ee84cc6bacf15584dcfd086; no voters: c53d47f2877d47dc9e0e6d6793562c71, ee6ba208af054a77a67f4e0598cd8c63
I20260929 10:35:50.625149 31000 raft_consensus.cc:2726] T 7e37caac011d47739dae44a915031421 P f379905b6ee84cc6bacf15584dcfd086 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 10:35:50.826069 31001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:50.825775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:37881 (local address 127.15.82.65:38059)
0929 10:35:50.825906 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:50.825979 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260929 10:35:50.972414 31005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:50.971972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:37881 (local address 127.15.82.66:37557)
0929 10:35:50.972245 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:50.972303 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":85,"threads_started":1}
I20260929 10:35:51.166669 31006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:51.166265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:37881 (local address 127.15.82.67:46811)
0929 10:35:51.166528 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:51.166582 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":72,"threads_started":1}
I20260929 10:35:51.827356 31007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:51.826880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:37881 (local address 127.15.82.65:49395)
0929 10:35:51.827183 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:51.827249 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":88,"threads_started":1}
I20260929 10:35:51.973755 31008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:51.973310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:37881 (local address 127.15.82.66:58523)
0929 10:35:51.973600 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:51.973661 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":92,"threads_started":1}
I20260929 10:35:52.167932 31009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:52.167407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:37881 (local address 127.15.82.67:54905)
0929 10:35:52.167737 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:52.167815 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":90,"threads_started":1}
I20260929 10:35:52.782969 31010 raft_consensus.cc:482] T 7e37caac011d47739dae44a915031421 P f379905b6ee84cc6bacf15584dcfd086 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:35:52.783108 31010 raft_consensus.cc:504] T 7e37caac011d47739dae44a915031421 P f379905b6ee84cc6bacf15584dcfd086 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c53d47f2877d47dc9e0e6d6793562c71" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 38923 } } peers { permanent_uuid: "ee6ba208af054a77a67f4e0598cd8c63" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 38555 } } peers { permanent_uuid: "f379905b6ee84cc6bacf15584dcfd086" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34003 } }
I20260929 10:35:52.783496 31010 leader_election.cc:288] T 7e37caac011d47739dae44a915031421 P f379905b6ee84cc6bacf15584dcfd086 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c53d47f2877d47dc9e0e6d6793562c71 (127.15.82.66:38923), ee6ba208af054a77a67f4e0598cd8c63 (127.15.82.67:38555)
W20260929 10:35:52.784387 30614 leader_election.cc:341] T 7e37caac011d47739dae44a915031421 P f379905b6ee84cc6bacf15584dcfd086 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ee6ba208af054a77a67f4e0598cd8c63 (127.15.82.67:38555): Not found: Tablet not found: 7e37caac011d47739dae44a915031421
W20260929 10:35:52.784478 30616 leader_election.cc:341] T 7e37caac011d47739dae44a915031421 P f379905b6ee84cc6bacf15584dcfd086 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c53d47f2877d47dc9e0e6d6793562c71 (127.15.82.66:38923): Not found: Tablet not found: 7e37caac011d47739dae44a915031421
I20260929 10:35:52.784590 30616 leader_election.cc:302] T 7e37caac011d47739dae44a915031421 P f379905b6ee84cc6bacf15584dcfd086 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f379905b6ee84cc6bacf15584dcfd086; no voters: c53d47f2877d47dc9e0e6d6793562c71, ee6ba208af054a77a67f4e0598cd8c63
I20260929 10:35:52.784773 31010 raft_consensus.cc:2726] T 7e37caac011d47739dae44a915031421 P f379905b6ee84cc6bacf15584dcfd086 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 10:35:52.828610 31011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:52.828146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:37881 (local address 127.15.82.65:47679)
0929 10:35:52.828444 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:52.828509 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":92,"threads_started":1}
I20260929 10:35:52.974974 31012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:52.974585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:37881 (local address 127.15.82.66:42843)
0929 10:35:52.974812 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:52.974864 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":75,"threads_started":1}
I20260929 10:35:53.169025 31013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:53.168642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:37881 (local address 127.15.82.67:53463)
0929 10:35:53.168877 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:53.168929 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":70,"threads_started":1}
I20260929 10:35:53.830160 31014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:53.829587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:37881 (local address 127.15.82.65:46977)
0929 10:35:53.829988 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:53.830056 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":118,"threads_started":1}
I20260929 10:35:53.976262 31015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:53.975852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:37881 (local address 127.15.82.66:39433)
0929 10:35:53.976112 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:53.976169 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":92,"threads_started":1}
I20260929 10:35:54.170250 31016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:54.169843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:37881 (local address 127.15.82.67:51593)
0929 10:35:54.170097 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:54.170159 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":73,"threads_started":1}
I20260929 10:35:54.831436 31017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:54.831004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:37881 (local address 127.15.82.65:51455)
0929 10:35:54.831269 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:54.831334 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":77,"threads_started":1}
I20260929 10:35:54.977427 31018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:54.976976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:37881 (local address 127.15.82.66:46659)
0929 10:35:54.977272 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:54.977329 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":87,"threads_started":1}
I20260929 10:35:55.171504 31019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:55.171076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:37881 (local address 127.15.82.67:46147)
0929 10:35:55.171349 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:55.171409 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":102,"threads_started":1}
I20260929 10:35:55.209271 31020 raft_consensus.cc:482] T 7e37caac011d47739dae44a915031421 P f379905b6ee84cc6bacf15584dcfd086 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:35:55.209398 31020 raft_consensus.cc:504] T 7e37caac011d47739dae44a915031421 P f379905b6ee84cc6bacf15584dcfd086 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c53d47f2877d47dc9e0e6d6793562c71" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 38923 } } peers { permanent_uuid: "ee6ba208af054a77a67f4e0598cd8c63" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 38555 } } peers { permanent_uuid: "f379905b6ee84cc6bacf15584dcfd086" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34003 } }
I20260929 10:35:55.209806 31020 leader_election.cc:288] T 7e37caac011d47739dae44a915031421 P f379905b6ee84cc6bacf15584dcfd086 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c53d47f2877d47dc9e0e6d6793562c71 (127.15.82.66:38923), ee6ba208af054a77a67f4e0598cd8c63 (127.15.82.67:38555)
W20260929 10:35:55.210632 30616 leader_election.cc:341] T 7e37caac011d47739dae44a915031421 P f379905b6ee84cc6bacf15584dcfd086 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c53d47f2877d47dc9e0e6d6793562c71 (127.15.82.66:38923): Not found: Tablet not found: 7e37caac011d47739dae44a915031421
W20260929 10:35:55.210804 30614 leader_election.cc:341] T 7e37caac011d47739dae44a915031421 P f379905b6ee84cc6bacf15584dcfd086 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ee6ba208af054a77a67f4e0598cd8c63 (127.15.82.67:38555): Not found: Tablet not found: 7e37caac011d47739dae44a915031421
I20260929 10:35:55.210891 30614 leader_election.cc:302] T 7e37caac011d47739dae44a915031421 P f379905b6ee84cc6bacf15584dcfd086 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f379905b6ee84cc6bacf15584dcfd086; no voters: c53d47f2877d47dc9e0e6d6793562c71, ee6ba208af054a77a67f4e0598cd8c63
I20260929 10:35:55.211076 31020 raft_consensus.cc:2726] T 7e37caac011d47739dae44a915031421 P f379905b6ee84cc6bacf15584dcfd086 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 10:35:55.832721 31021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:55.832299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:37881 (local address 127.15.82.65:58971)
0929 10:35:55.832572 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:55.832625 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":106,"threads_started":1}
I20260929 10:35:55.978791 31022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:55.978276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:37881 (local address 127.15.82.66:39867)
0929 10:35:55.978635 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:55.978695 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":95,"threads_started":1}
I20260929 10:35:56.172915 31023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:56.172391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:37881 (local address 127.15.82.67:52867)
0929 10:35:56.172727 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:56.172809 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":93,"threads_started":1}
I20260929 10:35:56.834182 31024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:56.833618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:37881 (local address 127.15.82.65:46845)
0929 10:35:56.833964 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:56.834055 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":95,"threads_started":1}
I20260929 10:35:56.980021 31025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:56.979562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:37881 (local address 127.15.82.66:43227)
0929 10:35:56.979832 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:56.979920 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":95,"threads_started":1}
I20260929 10:35:57.174226 31026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:57.173775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:37881 (local address 127.15.82.67:51443)
0929 10:35:57.174061 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:57.174126 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":95,"threads_started":1}
I20260929 10:35:57.835506 31027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:57.835071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:37881 (local address 127.15.82.65:49325)
0929 10:35:57.835345 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:57.835396 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":111,"threads_started":1}
I20260929 10:35:57.981323 31028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:57.980855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:37881 (local address 127.15.82.66:60045)
0929 10:35:57.981161 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:57.981225 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":81,"threads_started":1}
I20260929 10:35:58.175688 31029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:58.175177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:37881 (local address 127.15.82.67:39625)
0929 10:35:58.175487 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:58.175571 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":124,"threads_started":1}
I20260929 10:35:58.836951 31030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:58.836467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:37881 (local address 127.15.82.65:42165)
0929 10:35:58.836734 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:58.836814 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":77,"threads_started":1}
I20260929 10:35:58.982550 31031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:58.982088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:37881 (local address 127.15.82.66:52877)
0929 10:35:58.982330 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:58.982390 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":74,"threads_started":1}
I20260929 10:35:59.177060 31032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:59.176539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:37881 (local address 127.15.82.67:57505)
0929 10:35:59.176895 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:59.176962 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":90,"threads_started":1}
I20260929 10:35:59.838279 31033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:59.837768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:37881 (local address 127.15.82.65:34865)
0929 10:35:59.838128 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:59.838183 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":90,"threads_started":1}
I20260929 10:35:59.845039 31034 raft_consensus.cc:482] T 7e37caac011d47739dae44a915031421 P f379905b6ee84cc6bacf15584dcfd086 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:35:59.845181 31034 raft_consensus.cc:504] T 7e37caac011d47739dae44a915031421 P f379905b6ee84cc6bacf15584dcfd086 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c53d47f2877d47dc9e0e6d6793562c71" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 38923 } } peers { permanent_uuid: "ee6ba208af054a77a67f4e0598cd8c63" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 38555 } } peers { permanent_uuid: "f379905b6ee84cc6bacf15584dcfd086" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34003 } }
I20260929 10:35:59.845629 31034 leader_election.cc:288] T 7e37caac011d47739dae44a915031421 P f379905b6ee84cc6bacf15584dcfd086 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c53d47f2877d47dc9e0e6d6793562c71 (127.15.82.66:38923), ee6ba208af054a77a67f4e0598cd8c63 (127.15.82.67:38555)
W20260929 10:35:59.846457 30616 leader_election.cc:341] T 7e37caac011d47739dae44a915031421 P f379905b6ee84cc6bacf15584dcfd086 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c53d47f2877d47dc9e0e6d6793562c71 (127.15.82.66:38923): Not found: Tablet not found: 7e37caac011d47739dae44a915031421
W20260929 10:35:59.846544 30614 leader_election.cc:341] T 7e37caac011d47739dae44a915031421 P f379905b6ee84cc6bacf15584dcfd086 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ee6ba208af054a77a67f4e0598cd8c63 (127.15.82.67:38555): Not found: Tablet not found: 7e37caac011d47739dae44a915031421
I20260929 10:35:59.846589 30616 leader_election.cc:302] T 7e37caac011d47739dae44a915031421 P f379905b6ee84cc6bacf15584dcfd086 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f379905b6ee84cc6bacf15584dcfd086; no voters: c53d47f2877d47dc9e0e6d6793562c71, ee6ba208af054a77a67f4e0598cd8c63
I20260929 10:35:59.846818 31034 raft_consensus.cc:2726] T 7e37caac011d47739dae44a915031421 P f379905b6ee84cc6bacf15584dcfd086 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 10:35:59.983680 31035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:35:59.983275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:37881 (local address 127.15.82.66:33367)
0929 10:35:59.983526 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0929 10:35:59.983585 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":72,"threads_started":1}
I20260929 10:36:00.178303 31036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:00.177847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:37881 (local address 127.15.82.67:58973)
0929 10:36:00.178152 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0929 10:36:00.178209 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":79,"threads_started":1}
I20260929 10:36:00.839509 31037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:00.839081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:37881 (local address 127.15.82.65:46793)
0929 10:36:00.839346 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0929 10:36:00.839407 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":73,"threads_started":1}
I20260929 10:36:00.984793 31038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:00.984385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:37881 (local address 127.15.82.66:33341)
0929 10:36:00.984630 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0929 10:36:00.984688 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":84,"threads_started":1}
I20260929 10:36:01.179716 31039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:01.179228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:37881 (local address 127.15.82.67:43133)
0929 10:36:01.179508 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0929 10:36:01.179568 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":109,"threads_started":1}
I20260929 10:36:01.840837 31040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:01.840368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:37881 (local address 127.15.82.65:60193)
0929 10:36:01.840641 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0929 10:36:01.840696 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":97,"threads_started":1}
I20260929 10:36:01.986047 31041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:01.985626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:37881 (local address 127.15.82.66:44293)
0929 10:36:01.985890 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0929 10:36:01.985948 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":70,"threads_started":1}
I20260929 10:36:02.181171 31042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:02.180661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:37881 (local address 127.15.82.67:41433)
0929 10:36:02.181002 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0929 10:36:02.181070 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":93,"threads_started":1}
I20260929 10:36:02.842119 31043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:02.841582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:37881 (local address 127.15.82.65:37843)
0929 10:36:02.841954 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0929 10:36:02.842020 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":98,"threads_started":1}
I20260929 10:36:02.987418 31044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:02.986994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:37881 (local address 127.15.82.66:37361)
0929 10:36:02.987252 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0929 10:36:02.987319 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":77,"threads_started":1}
I20260929 10:36:03.182629 31045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:03.182033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:37881 (local address 127.15.82.67:40541)
0929 10:36:03.182377 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0929 10:36:03.182523 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":81,"threads_started":1}
I20260929 10:36:03.217455 31046 raft_consensus.cc:482] T 7e37caac011d47739dae44a915031421 P f379905b6ee84cc6bacf15584dcfd086 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:36:03.217651 31046 raft_consensus.cc:504] T 7e37caac011d47739dae44a915031421 P f379905b6ee84cc6bacf15584dcfd086 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c53d47f2877d47dc9e0e6d6793562c71" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 38923 } } peers { permanent_uuid: "ee6ba208af054a77a67f4e0598cd8c63" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 38555 } } peers { permanent_uuid: "f379905b6ee84cc6bacf15584dcfd086" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34003 } }
I20260929 10:36:03.218294 31046 leader_election.cc:288] T 7e37caac011d47739dae44a915031421 P f379905b6ee84cc6bacf15584dcfd086 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c53d47f2877d47dc9e0e6d6793562c71 (127.15.82.66:38923), ee6ba208af054a77a67f4e0598cd8c63 (127.15.82.67:38555)
W20260929 10:36:03.219056 30614 leader_election.cc:341] T 7e37caac011d47739dae44a915031421 P f379905b6ee84cc6bacf15584dcfd086 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ee6ba208af054a77a67f4e0598cd8c63 (127.15.82.67:38555): Not found: Tablet not found: 7e37caac011d47739dae44a915031421
W20260929 10:36:03.219197 30616 leader_election.cc:341] T 7e37caac011d47739dae44a915031421 P f379905b6ee84cc6bacf15584dcfd086 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c53d47f2877d47dc9e0e6d6793562c71 (127.15.82.66:38923): Not found: Tablet not found: 7e37caac011d47739dae44a915031421
I20260929 10:36:03.219295 30616 leader_election.cc:302] T 7e37caac011d47739dae44a915031421 P f379905b6ee84cc6bacf15584dcfd086 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f379905b6ee84cc6bacf15584dcfd086; no voters: c53d47f2877d47dc9e0e6d6793562c71, ee6ba208af054a77a67f4e0598cd8c63
I20260929 10:36:03.219473 31046 raft_consensus.cc:2726] T 7e37caac011d47739dae44a915031421 P f379905b6ee84cc6bacf15584dcfd086 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 10:36:03.843640 31047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:03.843171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:37881 (local address 127.15.82.65:45095)
0929 10:36:03.843470 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0929 10:36:03.843537 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":80,"threads_started":1}
I20260929 10:36:03.988776 31048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:03.988375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:37881 (local address 127.15.82.66:38927)
0929 10:36:03.988621 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0929 10:36:03.988678 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":83,"threads_started":1}
I20260929 10:36:04.172237 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 30563
I20260929 10:36:04.178215 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 30730
I20260929 10:36:04.184000 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 30864
I20260929 10:36:04.184016 31049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:04.183599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:37881 (local address 127.15.82.67:41659)
0929 10:36:04.183864 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0929 10:36:04.183922 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":96,"threads_started":1}
2026-09-29T10:36:04Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19662 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:36:04 dist-test-slave-vg89 krb5kdc[31052](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:36:04 dist-test-slave-vg89 krb5kdc[31052](info): set up 2 sockets
Sep 29 10:36:04 dist-test-slave-vg89 krb5kdc[31052](info): commencing operation
krb5kdc: starting...
W20260929 10:36:05.231563 15689 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Sep 29 10:36:05 dist-test-slave-vg89 krb5kdc[31052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678165, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:36:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:36:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.82.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab.
Entry for principal kudu/127.15.82.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab.
I20260929 10:36:05.361631 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:44709
--webserver_interface=127.15.82.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab
--principal=kudu/127.15.82.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:34999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.82.126:44709
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:36:05.477734 31065 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:36:05.478106 31065 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:36:05.478196 31065 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:36:05.482496 31065 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260929 10:36:05.482610 31065 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260929 10:36:05.482637 31065 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260929 10:36:05.482661 31065 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:36:05.482710 31065 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:36:05.482779 31065 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:36:05.482816 31065 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:36:05.482851 31065 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:36:05.488520 31065 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:34999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.15.82.126:44709
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab
--principal=kudu/127.15.82.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:44709
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.82.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31065
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:36:05.489760 31065 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:36:05.490901 31065 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 10:36:05.500479 31065 server_base.cc:1034] running on GCE node
W20260929 10:36:05.500428 31073 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:36:05.500415 31070 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:36:05.500420 31071 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:36:05.501309 31065 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:36:05.502589 31065 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:36:05.503806 31065 hybrid_clock.cc:648] HybridClock initialized: now 1790678165503787 us; error 35 us; skew 500 ppm
Sep 29 10:36:05 dist-test-slave-vg89 krb5kdc[31052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678165, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:36:05.506843 31065 init.cc:377] Logged in from keytab as kudu/127.15.82.126@KRBTEST.COM (short username kudu)
I20260929 10:36:05.507860 31065 webserver.cc:460] Webserver started at http://127.15.82.126:36519/ using document root <none> and password file <none>
I20260929 10:36:05.508487 31065 fs_manager.cc:359] Metadata directory not provided
I20260929 10:36:05.508553 31065 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:36:05.508800 31065 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:36:05.511039 31065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/master-0/data/instance:
uuid: "8fb3386c28a34308a393436c0d1cde01"
format_stamp: "Formatted at 2026-09-29 10:36:05 on dist-test-slave-vg89"
server_key: "1542023ca7ac7ab95c4e1ede5b9e973f"
server_key_iv: "939e30e658d7c6464f336ce80cee90ae"
server_key_version: "encryptionkey@0"
I20260929 10:36:05.511632 31065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/master-0/wal/instance:
uuid: "8fb3386c28a34308a393436c0d1cde01"
format_stamp: "Formatted at 2026-09-29 10:36:05 on dist-test-slave-vg89"
server_key: "1542023ca7ac7ab95c4e1ede5b9e973f"
server_key_iv: "939e30e658d7c6464f336ce80cee90ae"
server_key_version: "encryptionkey@0"
I20260929 10:36:05.515635 31065 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 10:36:05.518247 31080 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:36:05.519626 31065 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 10:36:05.519784 31065 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/master-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/master-0/wal
uuid: "8fb3386c28a34308a393436c0d1cde01"
format_stamp: "Formatted at 2026-09-29 10:36:05 on dist-test-slave-vg89"
server_key: "1542023ca7ac7ab95c4e1ede5b9e973f"
server_key_iv: "939e30e658d7c6464f336ce80cee90ae"
server_key_version: "encryptionkey@0"
I20260929 10:36:05.519909 31065 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:36:05.556823 31065 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:36:05.561625 31065 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:36:05.561885 31065 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:36:05.570216 31065 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.126:44709
I20260929 10:36:05.570588 31132 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.126:44709 every 8 connection(s)
I20260929 10:36:05.571448 31065 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
I20260929 10:36:05.574455 31133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:36:05.579680 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 31065
I20260929 10:36:05.579824 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/master-0/wal/instance
I20260929 10:36:05.580128 15689 external_mini_cluster.cc:1428] Setting key 3f6828168d865093766434f471b4bd15
I20260929 10:36:05.580322 31133 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fb3386c28a34308a393436c0d1cde01: Bootstrap starting.
I20260929 10:36:05.583058 31133 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8fb3386c28a34308a393436c0d1cde01: Neither blocks nor log segments found. Creating new log.
I20260929 10:36:05.583812 31133 log.cc:824] T 00000000000000000000000000000000 P 8fb3386c28a34308a393436c0d1cde01: Log is configured to *not* fsync() on all Append() calls
I20260929 10:36:05.585750 31133 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fb3386c28a34308a393436c0d1cde01: No bootstrap required, opened a new log
Sep 29 10:36:05 dist-test-slave-vg89 krb5kdc[31052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678165, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:36:05.589599 31133 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8fb3386c28a34308a393436c0d1cde01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fb3386c28a34308a393436c0d1cde01" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 44709 } }
I20260929 10:36:05.589856 31133 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8fb3386c28a34308a393436c0d1cde01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:36:05.589946 31133 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8fb3386c28a34308a393436c0d1cde01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fb3386c28a34308a393436c0d1cde01, State: Initialized, Role: FOLLOWER
I20260929 10:36:05.590513 31133 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8fb3386c28a34308a393436c0d1cde01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fb3386c28a34308a393436c0d1cde01" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 44709 } }
I20260929 10:36:05.590713 31133 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8fb3386c28a34308a393436c0d1cde01 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:36:05.590793 31133 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8fb3386c28a34308a393436c0d1cde01 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:36:05.590935 31133 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8fb3386c28a34308a393436c0d1cde01 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:36:05.591868 31133 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8fb3386c28a34308a393436c0d1cde01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fb3386c28a34308a393436c0d1cde01" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 44709 } }
I20260929 10:36:05.592284 31133 leader_election.cc:302] T 00000000000000000000000000000000 P 8fb3386c28a34308a393436c0d1cde01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8fb3386c28a34308a393436c0d1cde01; no voters:
I20260929 10:36:05.592691 31133 leader_election.cc:288] T 00000000000000000000000000000000 P 8fb3386c28a34308a393436c0d1cde01 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:36:05.592813 31138 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8fb3386c28a34308a393436c0d1cde01 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:36:05.593039 31138 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8fb3386c28a34308a393436c0d1cde01 [term 1 LEADER]: Becoming Leader. State: Replica: 8fb3386c28a34308a393436c0d1cde01, State: Running, Role: LEADER
I20260929 10:36:05.593389 31138 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8fb3386c28a34308a393436c0d1cde01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fb3386c28a34308a393436c0d1cde01" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 44709 } }
I20260929 10:36:05.594089 31133 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8fb3386c28a34308a393436c0d1cde01 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:36:05.595255 31139 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fb3386c28a34308a393436c0d1cde01 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8fb3386c28a34308a393436c0d1cde01" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fb3386c28a34308a393436c0d1cde01" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 44709 } } }
I20260929 10:36:05.595256 31140 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fb3386c28a34308a393436c0d1cde01 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8fb3386c28a34308a393436c0d1cde01. Latest consensus state: current_term: 1 leader_uuid: "8fb3386c28a34308a393436c0d1cde01" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fb3386c28a34308a393436c0d1cde01" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 44709 } } }
I20260929 10:36:05.595417 31139 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fb3386c28a34308a393436c0d1cde01 [sys.catalog]: This master's current role is: LEADER
I20260929 10:36:05.595417 31140 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fb3386c28a34308a393436c0d1cde01 [sys.catalog]: This master's current role is: LEADER
I20260929 10:36:05.595674 31136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:05.581507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59280 (local address 127.15.82.126:44709)
0929 10:36:05.582015 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0929 10:36:05.582031 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0929 10:36:05.582068 (+ 37us) server_negotiation.cc:407] Connection header received
0929 10:36:05.582805 (+ 737us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:36:05.582830 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:36:05.583210 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:36:05.583586 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:36:05.584417 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:36:05.585447 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:36:05.586439 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:36:05.586936 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:36:05.587122 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:36:05.589615 (+ 2493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:36:05.589620 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:36:05.589634 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:36:05.589662 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:36:05.591643 (+ 1981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:36:05.592135 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:36:05.592141 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:36:05.592147 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:36:05.592295 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:36:05.592659 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:36:05.592662 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:36:05.592664 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:36:05.593196 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:36:05.594005 (+ 809us) server_negotiation.cc:1035] Waiting for connection context
0929 10:36:05.594173 (+ 168us) server_negotiation.cc:299] Negotiation successful
0929 10:36:05.594544 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":166,"threads_started":1}
I20260929 10:36:05.595849 31147 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:36:05.599220 31147 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:36:05.604346 31147 catalog_manager.cc:1347] Generated new cluster ID: b3abbf64430d41ac8eec98adc906db15
I20260929 10:36:05.604421 31147 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:36:05.612547 31147 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:36:05.613796 31147 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:36:05.626222 31147 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8fb3386c28a34308a393436c0d1cde01: Generated new TSK 0
I20260929 10:36:05.627063 31147 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.82.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab.
Entry for principal kudu/127.15.82.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab.
I20260929 10:36:05.666143 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.65:0
--local_ip_for_outbound_sockets=127.15.82.65
--webserver_interface=127.15.82.65
--webserver_port=0
--tserver_master_addrs=127.15.82.126:44709
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab
--principal=kudu/127.15.82.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:34999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:36:05.784961 31159 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:36:05.785305 31159 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:36:05.785360 31159 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:36:05.789593 31159 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:36:05.789670 31159 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:36:05.789736 31159 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.65
I20260929 10:36:05.795387 31159 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:34999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab
--principal=kudu/127.15.82.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.82.65
--webserver_port=0
--tserver_master_addrs=127.15.82.126:44709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31159
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.65
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:36:05.796593 31159 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:36:05.797642 31159 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:36:05.807930 31165 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:36:05.807951 31167 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:36:05.808061 31164 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:36:05.808210 31159 server_base.cc:1034] running on GCE node
I20260929 10:36:05.808619 31159 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:36:05.809362 31159 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:36:05.810556 31159 hybrid_clock.cc:648] HybridClock initialized: now 1790678165810536 us; error 36 us; skew 500 ppm
Sep 29 10:36:05 dist-test-slave-vg89 krb5kdc[31052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678165, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:36:05.813470 31159 init.cc:377] Logged in from keytab as kudu/127.15.82.65@KRBTEST.COM (short username kudu)
I20260929 10:36:05.814579 31159 webserver.cc:460] Webserver started at http://127.15.82.65:36785/ using document root <none> and password file <none>
I20260929 10:36:05.815186 31159 fs_manager.cc:359] Metadata directory not provided
I20260929 10:36:05.815286 31159 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:36:05.815529 31159 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:36:05.817736 31159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-0/data/instance:
uuid: "63f8e03d49954851a2b4b48305144ee5"
format_stamp: "Formatted at 2026-09-29 10:36:05 on dist-test-slave-vg89"
server_key: "3572e645b2fd051b89fa5a6e2866f4ac"
server_key_iv: "3a2d499a1385e60427468602d7a17cdc"
server_key_version: "encryptionkey@0"
I20260929 10:36:05.818377 31159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-0/wal/instance:
uuid: "63f8e03d49954851a2b4b48305144ee5"
format_stamp: "Formatted at 2026-09-29 10:36:05 on dist-test-slave-vg89"
server_key: "3572e645b2fd051b89fa5a6e2866f4ac"
server_key_iv: "3a2d499a1385e60427468602d7a17cdc"
server_key_version: "encryptionkey@0"
I20260929 10:36:05.822710 31159 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260929 10:36:05.825472 31174 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:36:05.826862 31159 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 10:36:05.827046 31159 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-0/wal
uuid: "63f8e03d49954851a2b4b48305144ee5"
format_stamp: "Formatted at 2026-09-29 10:36:05 on dist-test-slave-vg89"
server_key: "3572e645b2fd051b89fa5a6e2866f4ac"
server_key_iv: "3a2d499a1385e60427468602d7a17cdc"
server_key_version: "encryptionkey@0"
I20260929 10:36:05.827198 31159 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:36:05.852950 31159 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:36:05.856138 31159 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:36:05.856379 31159 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:36:05.857051 31159 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:36:05.857966 31159 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:36:05.858044 31159 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:36:05.858124 31159 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:36:05.858170 31159 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:36:05.868363 31159 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.65:36197
I20260929 10:36:05.868386 31287 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.65:36197 every 8 connection(s)
I20260929 10:36:05.869377 31159 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
I20260929 10:36:05.873847 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 31159
I20260929 10:36:05.874027 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-0/wal/instance
I20260929 10:36:05.874354 15689 external_mini_cluster.cc:1428] Setting key 1f58cc6f98d72f31a3d07044024cde86
Sep 29 10:36:05 dist-test-slave-vg89 krb5kdc[31052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678165, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.65@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:36:05.884748 31136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:05.871369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.65:50223 (local address 127.15.82.126:44709)
0929 10:36:05.871519 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0929 10:36:05.871523 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:36:05.872564 (+ 1041us) server_negotiation.cc:407] Connection header received
0929 10:36:05.873380 (+ 816us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:36:05.873382 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:36:05.873435 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:36:05.873521 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:36:05.874898 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:36:05.875565 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:36:05.876832 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:36:05.877245 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:36:05.877371 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:36:05.880393 (+ 3022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:36:05.880399 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:36:05.880404 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:36:05.880435 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:36:05.882322 (+ 1887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:36:05.882923 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:36:05.882929 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:36:05.882932 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:36:05.883030 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:36:05.883430 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:36:05.883436 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:36:05.883438 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:36:05.883701 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:36:05.883812 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0929 10:36:05.884364 (+ 552us) server_negotiation.cc:299] Negotiation successful
0929 10:36:05.884542 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
WARNING: no policy specified for kudu/127.15.82.66@KRBTEST.COM; defaulting to no policy
I20260929 10:36:05.885506 31290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:05.871732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:44709 (local address 127.15.82.65:50223)
0929 10:36:05.872385 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0929 10:36:05.872427 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 10:36:05.873190 (+ 763us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:36:05.873708 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:36:05.873715 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:36:05.874156 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:36:05.874724 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:36:05.874739 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:36:05.875747 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:36:05.875751 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:36:05.876686 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:36:05.876697 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:36:05.877408 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:36:05.877411 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:36:05.877572 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:36:05.878370 (+ 798us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:36:05.878388 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:36:05.880202 (+ 1814us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:36:05.882507 (+ 2305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:36:05.882512 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:36:05.882525 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:36:05.882807 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:36:05.883134 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:36:05.883137 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:36:05.883138 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:36:05.883301 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:36:05.883836 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:36:05.883841 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:36:05.884091 (+ 250us) client_negotiation.cc:769] Sending connection context
0929 10:36:05.884256 (+ 165us) client_negotiation.cc:240] Negotiation successful
0929 10:36:05.884528 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":118,"threads_started":1}
I20260929 10:36:05.886922 31288 heartbeater.cc:347] Connected to a master server at 127.15.82.126:44709
I20260929 10:36:05.887207 31288 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.66@KRBTEST.COM" created.
I20260929 10:36:05.887852 31288 heartbeater.cc:511] Master 127.15.82.126:44709 requested a full tablet report, sending...
I20260929 10:36:05.889604 31090 ts_manager.cc:194] Registered new tserver with Master: 63f8e03d49954851a2b4b48305144ee5 (127.15.82.65:36197)
I20260929 10:36:05.890836 31090 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.82.65@KRBTEST.COM'} at 127.15.82.65:50223
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab.
Entry for principal kudu/127.15.82.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab.
I20260929 10:36:05.905067 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.66:0
--local_ip_for_outbound_sockets=127.15.82.66
--webserver_interface=127.15.82.66
--webserver_port=0
--tserver_master_addrs=127.15.82.126:44709
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab
--principal=kudu/127.15.82.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:34999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:36:06.018190 31293 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:36:06.018525 31293 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:36:06.018647 31293 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:36:06.022657 31293 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:36:06.022708 31293 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:36:06.022783 31293 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.66
I20260929 10:36:06.028129 31293 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:34999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab
--principal=kudu/127.15.82.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.82.66
--webserver_port=0
--tserver_master_addrs=127.15.82.126:44709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31293
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.66
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:36:06.029196 31293 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:36:06.030290 31293 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:36:06.040126 31299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:36:06.040158 31298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:36:06.040407 31293 server_base.cc:1034] running on GCE node
W20260929 10:36:06.040126 31301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:36:06.040843 31293 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:36:06.041460 31293 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:36:06.042614 31293 hybrid_clock.cc:648] HybridClock initialized: now 1790678166042599 us; error 26 us; skew 500 ppm
Sep 29 10:36:06 dist-test-slave-vg89 krb5kdc[31052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678166, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:36:06.045252 31293 init.cc:377] Logged in from keytab as kudu/127.15.82.66@KRBTEST.COM (short username kudu)
I20260929 10:36:06.046146 31293 webserver.cc:460] Webserver started at http://127.15.82.66:33533/ using document root <none> and password file <none>
I20260929 10:36:06.046773 31293 fs_manager.cc:359] Metadata directory not provided
I20260929 10:36:06.046864 31293 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:36:06.047099 31293 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:36:06.049094 31293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-1/data/instance:
uuid: "e6581ebffa3246a987710e539ce43fa3"
format_stamp: "Formatted at 2026-09-29 10:36:06 on dist-test-slave-vg89"
server_key: "0bfe256a079924566e473b8a3bd241f8"
server_key_iv: "8d5aeb5922cc2793b1b2e4164c640eff"
server_key_version: "encryptionkey@0"
I20260929 10:36:06.049669 31293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-1/wal/instance:
uuid: "e6581ebffa3246a987710e539ce43fa3"
format_stamp: "Formatted at 2026-09-29 10:36:06 on dist-test-slave-vg89"
server_key: "0bfe256a079924566e473b8a3bd241f8"
server_key_iv: "8d5aeb5922cc2793b1b2e4164c640eff"
server_key_version: "encryptionkey@0"
I20260929 10:36:06.053280 31293 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260929 10:36:06.056071 31308 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:36:06.057292 31293 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260929 10:36:06.057427 31293 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-1/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-1/wal
uuid: "e6581ebffa3246a987710e539ce43fa3"
format_stamp: "Formatted at 2026-09-29 10:36:06 on dist-test-slave-vg89"
server_key: "0bfe256a079924566e473b8a3bd241f8"
server_key_iv: "8d5aeb5922cc2793b1b2e4164c640eff"
server_key_version: "encryptionkey@0"
I20260929 10:36:06.057548 31293 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:36:06.093614 31293 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:36:06.097003 31293 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:36:06.097203 31293 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:36:06.097795 31293 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:36:06.098709 31293 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:36:06.098780 31293 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:36:06.098852 31293 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:36:06.098902 31293 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:36:06.107630 31293 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.66:35159
I20260929 10:36:06.107654 31421 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.66:35159 every 8 connection(s)
I20260929 10:36:06.108480 31293 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
I20260929 10:36:06.112423 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 31293
I20260929 10:36:06.112552 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-1/wal/instance
I20260929 10:36:06.112869 15689 external_mini_cluster.cc:1428] Setting key 21d40f402db30e7c446d11a011f86bd2
Sep 29 10:36:06 dist-test-slave-vg89 krb5kdc[31052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678166, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.66@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.82.67@KRBTEST.COM; defaulting to no policy
I20260929 10:36:06.123576 31136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:06.110279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:44903 (local address 127.15.82.126:44709)
0929 10:36:06.110448 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0929 10:36:06.110454 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 10:36:06.111508 (+ 1054us) server_negotiation.cc:407] Connection header received
0929 10:36:06.112534 (+ 1026us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:36:06.112538 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:36:06.112604 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:36:06.112694 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:36:06.114143 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:36:06.114794 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:36:06.116574 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:36:06.116970 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:36:06.117100 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:36:06.119811 (+ 2711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:36:06.119815 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:36:06.119818 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:36:06.119849 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:36:06.121315 (+ 1466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:36:06.121839 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:36:06.121843 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:36:06.121845 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:36:06.121926 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:36:06.122283 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:36:06.122286 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:36:06.122288 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:36:06.122540 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:36:06.122643 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0929 10:36:06.123272 (+ 629us) server_negotiation.cc:299] Negotiation successful
0929 10:36:06.123405 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260929 10:36:06.124519 31424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:06.110745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:44709 (local address 127.15.82.66:44903)
0929 10:36:06.111363 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0929 10:36:06.111405 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 10:36:06.112284 (+ 879us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:36:06.112920 (+ 636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:36:06.112929 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:36:06.113419 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:36:06.113993 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:36:06.114012 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:36:06.114959 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:36:06.114965 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:36:06.116447 (+ 1482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:36:06.116457 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:36:06.117130 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:36:06.117133 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:36:06.117281 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:36:06.117874 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:36:06.117899 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:36:06.119646 (+ 1747us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:36:06.121446 (+ 1800us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:36:06.121451 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:36:06.121468 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:36:06.121732 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:36:06.122026 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:36:06.122029 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:36:06.122032 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:36:06.122178 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:36:06.122653 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:36:06.122662 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:36:06.122956 (+ 294us) client_negotiation.cc:769] Sending connection context
0929 10:36:06.123210 (+ 254us) client_negotiation.cc:240] Negotiation successful
0929 10:36:06.123567 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":106,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.67@KRBTEST.COM" created.
I20260929 10:36:06.125763 31422 heartbeater.cc:347] Connected to a master server at 127.15.82.126:44709
I20260929 10:36:06.126067 31422 heartbeater.cc:464] Registering TS with master...
I20260929 10:36:06.126741 31422 heartbeater.cc:511] Master 127.15.82.126:44709 requested a full tablet report, sending...
I20260929 10:36:06.128089 31090 ts_manager.cc:194] Registered new tserver with Master: e6581ebffa3246a987710e539ce43fa3 (127.15.82.66:35159)
I20260929 10:36:06.128669 31090 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.82.66@KRBTEST.COM'} at 127.15.82.66:44903
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab.
Entry for principal kudu/127.15.82.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab.
I20260929 10:36:06.141562 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.67:0
--local_ip_for_outbound_sockets=127.15.82.67
--webserver_interface=127.15.82.67
--webserver_port=0
--tserver_master_addrs=127.15.82.126:44709
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab
--principal=kudu/127.15.82.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:34999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:36:06.263517 31427 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:36:06.263851 31427 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:36:06.263931 31427 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:36:06.268366 31427 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:36:06.268462 31427 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:36:06.268560 31427 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.67
I20260929 10:36:06.273898 31427 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:34999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab
--principal=kudu/127.15.82.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.82.67
--webserver_port=0
--tserver_master_addrs=127.15.82.126:44709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31427
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.67
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:36:06.274991 31427 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:36:06.276036 31427 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:36:06.285781 31435 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:36:06.285779 31432 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:36:06.285792 31433 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:36:06.286150 31427 server_base.cc:1034] running on GCE node
I20260929 10:36:06.286631 31427 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:36:06.287258 31427 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:36:06.288516 31427 hybrid_clock.cc:648] HybridClock initialized: now 1790678166288493 us; error 32 us; skew 500 ppm
Sep 29 10:36:06 dist-test-slave-vg89 krb5kdc[31052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678166, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:36:06.291435 31427 init.cc:377] Logged in from keytab as kudu/127.15.82.67@KRBTEST.COM (short username kudu)
I20260929 10:36:06.292277 31427 webserver.cc:460] Webserver started at http://127.15.82.67:36695/ using document root <none> and password file <none>
I20260929 10:36:06.292826 31427 fs_manager.cc:359] Metadata directory not provided
I20260929 10:36:06.292887 31427 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:36:06.293099 31427 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:36:06.295292 31427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-2/data/instance:
uuid: "04ba93cc78414e468e3bb02139bc71c0"
format_stamp: "Formatted at 2026-09-29 10:36:06 on dist-test-slave-vg89"
server_key: "91fdf326a1efd5e729607d530e8a9406"
server_key_iv: "50bd17be0b3dda8ae4f58dabbeaebb19"
server_key_version: "encryptionkey@0"
I20260929 10:36:06.295882 31427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-2/wal/instance:
uuid: "04ba93cc78414e468e3bb02139bc71c0"
format_stamp: "Formatted at 2026-09-29 10:36:06 on dist-test-slave-vg89"
server_key: "91fdf326a1efd5e729607d530e8a9406"
server_key_iv: "50bd17be0b3dda8ae4f58dabbeaebb19"
server_key_version: "encryptionkey@0"
I20260929 10:36:06.299598 31427 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260929 10:36:06.301980 31442 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:36:06.303261 31427 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260929 10:36:06.303381 31427 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-2/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-2/wal
uuid: "04ba93cc78414e468e3bb02139bc71c0"
format_stamp: "Formatted at 2026-09-29 10:36:06 on dist-test-slave-vg89"
server_key: "91fdf326a1efd5e729607d530e8a9406"
server_key_iv: "50bd17be0b3dda8ae4f58dabbeaebb19"
server_key_version: "encryptionkey@0"
I20260929 10:36:06.303488 31427 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:36:06.331923 31427 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:36:06.335402 31427 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:36:06.335615 31427 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:36:06.336228 31427 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:36:06.337061 31427 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:36:06.337107 31427 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:36:06.337170 31427 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:36:06.337203 31427 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:36:06.346164 31427 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.67:44377
I20260929 10:36:06.346185 31555 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.67:44377 every 8 connection(s)
I20260929 10:36:06.347174 31427 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
I20260929 10:36:06.348865 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 31427
I20260929 10:36:06.348989 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-2/wal/instance
I20260929 10:36:06.349296 15689 external_mini_cluster.cc:1428] Setting key bbd7d90c8bc5ffcd034a577924a0be2c
Sep 29 10:36:06 dist-test-slave-vg89 krb5kdc[31052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678166, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.67@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:36:06.360749 31136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:06.348991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.67:60335 (local address 127.15.82.126:44709)
0929 10:36:06.349137 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0929 10:36:06.349148 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 10:36:06.350174 (+ 1026us) server_negotiation.cc:407] Connection header received
0929 10:36:06.351696 (+ 1522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:36:06.351699 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:36:06.351756 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:36:06.351832 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:36:06.353010 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:36:06.353552 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:36:06.354756 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:36:06.355000 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:36:06.355114 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:36:06.357443 (+ 2329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:36:06.357446 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:36:06.357448 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:36:06.357473 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:36:06.358958 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:36:06.359417 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:36:06.359419 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:36:06.359420 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:36:06.359494 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:36:06.359780 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:36:06.359782 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:36:06.359783 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:36:06.359949 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:36:06.360035 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0929 10:36:06.360475 (+ 440us) server_negotiation.cc:299] Negotiation successful
0929 10:36:06.360573 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260929 10:36:06.361585 31558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:06.349505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:44709 (local address 127.15.82.67:60335)
0929 10:36:06.350034 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0929 10:36:06.350086 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0929 10:36:06.351476 (+ 1390us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:36:06.352013 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:36:06.352020 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:36:06.352375 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:36:06.352858 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:36:06.352871 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:36:06.353682 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:36:06.353686 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:36:06.354609 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:36:06.354619 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:36:06.355150 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:36:06.355154 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:36:06.355327 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:36:06.355837 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:36:06.355875 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:36:06.357279 (+ 1404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:36:06.359074 (+ 1795us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:36:06.359079 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:36:06.359091 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:36:06.359328 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:36:06.359572 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:36:06.359574 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:36:06.359575 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:36:06.359701 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:36:06.360034 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:36:06.360039 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:36:06.360267 (+ 228us) client_negotiation.cc:769] Sending connection context
0929 10:36:06.360427 (+ 160us) client_negotiation.cc:240] Negotiation successful
0929 10:36:06.360762 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":161,"threads_started":1}
I20260929 10:36:06.362892 31556 heartbeater.cc:347] Connected to a master server at 127.15.82.126:44709
I20260929 10:36:06.363144 31556 heartbeater.cc:464] Registering TS with master...
I20260929 10:36:06.363677 31556 heartbeater.cc:511] Master 127.15.82.126:44709 requested a full tablet report, sending...
I20260929 10:36:06.364687 31090 ts_manager.cc:194] Registered new tserver with Master: 04ba93cc78414e468e3bb02139bc71c0 (127.15.82.67:44377)
I20260929 10:36:06.365247 31090 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.82.67@KRBTEST.COM'} at 127.15.82.67:60335
I20260929 10:36:06.375079 15689 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 10:36:06.384374 31136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:06.377182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59286 (local address 127.15.82.126:44709)
0929 10:36:06.377342 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0929 10:36:06.377347 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:36:06.377499 (+ 152us) server_negotiation.cc:407] Connection header received
0929 10:36:06.377568 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:36:06.377570 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:36:06.377609 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:36:06.377689 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:36:06.378341 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:36:06.378955 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:36:06.379750 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:36:06.379988 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:36:06.380060 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:36:06.381390 (+ 1330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:36:06.381393 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:36:06.381395 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:36:06.381412 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:36:06.382728 (+ 1316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:36:06.383102 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:36:06.383105 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:36:06.383106 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:36:06.383181 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:36:06.383437 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:36:06.383440 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:36:06.383441 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:36:06.383691 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:36:06.383843 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0929 10:36:06.384013 (+ 170us) server_negotiation.cc:299] Negotiation successful
0929 10:36:06.384159 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260929 10:36:06.387306 31090 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59286:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 10:36:06.390331 31090 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:36:06.405398 31567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:06.398845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.66:35159 (local address 127.0.0.1:53244)
0929 10:36:06.399257 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0929 10:36:06.399303 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0929 10:36:06.399441 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:36:06.400899 (+ 1458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:36:06.400905 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:36:06.400926 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:36:06.401083 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:36:06.401092 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:36:06.402328 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:36:06.402331 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:36:06.404028 (+ 1697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:36:06.404036 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:36:06.404827 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:36:06.404830 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:36:06.404953 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:36:06.404986 (+ 33us) client_negotiation.cc:769] Sending connection context
0929 10:36:06.405132 (+ 146us) client_negotiation.cc:240] Negotiation successful
0929 10:36:06.405253 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"mutex_wait_us":112,"thread_start_us":62,"threads_started":1}
I20260929 10:36:06.405406 31568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:06.398967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53244 (local address 127.15.82.66:35159)
0929 10:36:06.400397 (+ 1430us) server_negotiation.cc:206] Beginning negotiation
0929 10:36:06.400417 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0929 10:36:06.400435 (+ 18us) server_negotiation.cc:407] Connection header received
0929 10:36:06.400526 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:36:06.400531 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:36:06.400724 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:36:06.400907 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:36:06.401224 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:36:06.402202 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:36:06.404157 (+ 1955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:36:06.404714 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:36:06.404801 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:36:06.404914 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0929 10:36:06.405166 (+ 252us) server_negotiation.cc:299] Negotiation successful
0929 10:36:06.405264 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1329,"thread_start_us":136,"threads_started":1}
I20260929 10:36:06.406132 31569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:06.399543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.67:44377 (local address 127.0.0.1:46038)
0929 10:36:06.399801 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0929 10:36:06.399812 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:36:06.399883 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:36:06.401701 (+ 1818us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:36:06.401704 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:36:06.401724 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:36:06.401809 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:36:06.401815 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:36:06.403154 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:36:06.403158 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:36:06.404037 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:36:06.404046 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:36:06.405754 (+ 1708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:36:06.405758 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:36:06.405846 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:36:06.405868 (+ 22us) client_negotiation.cc:769] Sending connection context
0929 10:36:06.405939 (+ 71us) client_negotiation.cc:240] Negotiation successful
0929 10:36:06.406000 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"mutex_wait_us":25,"thread_start_us":110,"threads_started":1}
I20260929 10:36:06.406585 31570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:06.399727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46038 (local address 127.15.82.67:44377)
0929 10:36:06.401241 (+ 1514us) server_negotiation.cc:206] Beginning negotiation
0929 10:36:06.401277 (+ 36us) server_negotiation.cc:399] Waiting for connection header
0929 10:36:06.401299 (+ 22us) server_negotiation.cc:407] Connection header received
0929 10:36:06.401372 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:36:06.401377 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:36:06.401549 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:36:06.401691 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:36:06.401930 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:36:06.403008 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:36:06.404961 (+ 1953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:36:06.405632 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:36:06.405731 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:36:06.405880 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0929 10:36:06.406308 (+ 428us) server_negotiation.cc:299] Negotiation successful
0929 10:36:06.406447 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1372,"thread_start_us":144,"threads_started":1}
I20260929 10:36:06.407339 31571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:06.399948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.65:36197 (local address 127.0.0.1:48264)
0929 10:36:06.400154 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0929 10:36:06.400167 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 10:36:06.400245 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:36:06.400989 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:36:06.400992 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:36:06.401010 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:36:06.401135 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:36:06.401142 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:36:06.402629 (+ 1487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:36:06.402633 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:36:06.404025 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:36:06.404036 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:36:06.406948 (+ 2912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:36:06.406952 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:36:06.407054 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:36:06.407077 (+ 23us) client_negotiation.cc:769] Sending connection context
0929 10:36:06.407157 (+ 80us) client_negotiation.cc:240] Negotiation successful
0929 10:36:06.407221 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":99,"thread_start_us":87,"threads_started":1}
I20260929 10:36:06.407372 31572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:06.400140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48264 (local address 127.15.82.65:36197)
0929 10:36:06.400601 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0929 10:36:06.400608 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 10:36:06.400631 (+ 23us) server_negotiation.cc:407] Connection header received
0929 10:36:06.400712 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:36:06.400715 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:36:06.400861 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:36:06.401023 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:36:06.401301 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:36:06.402503 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:36:06.405940 (+ 3437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:36:06.406844 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:36:06.406949 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:36:06.407055 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0929 10:36:06.407187 (+ 132us) server_negotiation.cc:299] Negotiation successful
0929 10:36:06.407252 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":117,"threads_started":1}
I20260929 10:36:06.408767 31356 tablet_service.cc:1467] Processing CreateTablet for tablet 81048e496c6e4266861b399223be6a54 (DEFAULT_TABLE table=test-table [id=bb7e6acae25f4b819b42b2820d6424b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:36:06.409732 31356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81048e496c6e4266861b399223be6a54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:36:06.409911 31222 tablet_service.cc:1467] Processing CreateTablet for tablet 81048e496c6e4266861b399223be6a54 (DEFAULT_TABLE table=test-table [id=bb7e6acae25f4b819b42b2820d6424b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:36:06.410964 31222 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81048e496c6e4266861b399223be6a54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:36:06.417191 31574 tablet_bootstrap.cc:492] T 81048e496c6e4266861b399223be6a54 P 63f8e03d49954851a2b4b48305144ee5: Bootstrap starting.
I20260929 10:36:06.417402 31490 tablet_service.cc:1467] Processing CreateTablet for tablet 81048e496c6e4266861b399223be6a54 (DEFAULT_TABLE table=test-table [id=bb7e6acae25f4b819b42b2820d6424b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:36:06.418295 31490 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81048e496c6e4266861b399223be6a54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:36:06.419214 31574 tablet_bootstrap.cc:654] T 81048e496c6e4266861b399223be6a54 P 63f8e03d49954851a2b4b48305144ee5: Neither blocks nor log segments found. Creating new log.
I20260929 10:36:06.419907 31574 log.cc:824] T 81048e496c6e4266861b399223be6a54 P 63f8e03d49954851a2b4b48305144ee5: Log is configured to *not* fsync() on all Append() calls
I20260929 10:36:06.420621 31573 tablet_bootstrap.cc:492] T 81048e496c6e4266861b399223be6a54 P e6581ebffa3246a987710e539ce43fa3: Bootstrap starting.
I20260929 10:36:06.421972 31574 tablet_bootstrap.cc:492] T 81048e496c6e4266861b399223be6a54 P 63f8e03d49954851a2b4b48305144ee5: No bootstrap required, opened a new log
I20260929 10:36:06.422165 31574 ts_tablet_manager.cc:1397] T 81048e496c6e4266861b399223be6a54 P 63f8e03d49954851a2b4b48305144ee5: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260929 10:36:06.423144 31573 tablet_bootstrap.cc:654] T 81048e496c6e4266861b399223be6a54 P e6581ebffa3246a987710e539ce43fa3: Neither blocks nor log segments found. Creating new log.
I20260929 10:36:06.423907 31573 log.cc:824] T 81048e496c6e4266861b399223be6a54 P e6581ebffa3246a987710e539ce43fa3: Log is configured to *not* fsync() on all Append() calls
I20260929 10:36:06.425740 31573 tablet_bootstrap.cc:492] T 81048e496c6e4266861b399223be6a54 P e6581ebffa3246a987710e539ce43fa3: No bootstrap required, opened a new log
I20260929 10:36:06.425940 31573 ts_tablet_manager.cc:1397] T 81048e496c6e4266861b399223be6a54 P e6581ebffa3246a987710e539ce43fa3: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260929 10:36:06.426375 31574 raft_consensus.cc:348] T 81048e496c6e4266861b399223be6a54 P 63f8e03d49954851a2b4b48305144ee5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6581ebffa3246a987710e539ce43fa3" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 35159 } } peers { permanent_uuid: "04ba93cc78414e468e3bb02139bc71c0" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 44377 } } peers { permanent_uuid: "63f8e03d49954851a2b4b48305144ee5" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 36197 } }
I20260929 10:36:06.426661 31574 raft_consensus.cc:374] T 81048e496c6e4266861b399223be6a54 P 63f8e03d49954851a2b4b48305144ee5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:36:06.426740 31574 raft_consensus.cc:729] T 81048e496c6e4266861b399223be6a54 P 63f8e03d49954851a2b4b48305144ee5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63f8e03d49954851a2b4b48305144ee5, State: Initialized, Role: FOLLOWER
I20260929 10:36:06.427055 31577 tablet_bootstrap.cc:492] T 81048e496c6e4266861b399223be6a54 P 04ba93cc78414e468e3bb02139bc71c0: Bootstrap starting.
I20260929 10:36:06.427202 31574 consensus_queue.cc:260] T 81048e496c6e4266861b399223be6a54 P 63f8e03d49954851a2b4b48305144ee5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6581ebffa3246a987710e539ce43fa3" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 35159 } } peers { permanent_uuid: "04ba93cc78414e468e3bb02139bc71c0" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 44377 } } peers { permanent_uuid: "63f8e03d49954851a2b4b48305144ee5" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 36197 } }
I20260929 10:36:06.428241 31574 ts_tablet_manager.cc:1428] T 81048e496c6e4266861b399223be6a54 P 63f8e03d49954851a2b4b48305144ee5: Time spent starting tablet: real 0.006s user 0.004s sys 0.003s
I20260929 10:36:06.428630 31288 heartbeater.cc:503] Master 127.15.82.126:44709 was elected leader, sending a full tablet report...
I20260929 10:36:06.429129 31577 tablet_bootstrap.cc:654] T 81048e496c6e4266861b399223be6a54 P 04ba93cc78414e468e3bb02139bc71c0: Neither blocks nor log segments found. Creating new log.
I20260929 10:36:06.429762 31573 raft_consensus.cc:348] T 81048e496c6e4266861b399223be6a54 P e6581ebffa3246a987710e539ce43fa3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6581ebffa3246a987710e539ce43fa3" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 35159 } } peers { permanent_uuid: "04ba93cc78414e468e3bb02139bc71c0" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 44377 } } peers { permanent_uuid: "63f8e03d49954851a2b4b48305144ee5" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 36197 } }
I20260929 10:36:06.430181 31573 raft_consensus.cc:374] T 81048e496c6e4266861b399223be6a54 P e6581ebffa3246a987710e539ce43fa3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:36:06.430264 31573 raft_consensus.cc:729] T 81048e496c6e4266861b399223be6a54 P e6581ebffa3246a987710e539ce43fa3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6581ebffa3246a987710e539ce43fa3, State: Initialized, Role: FOLLOWER
I20260929 10:36:06.430323 31577 log.cc:824] T 81048e496c6e4266861b399223be6a54 P 04ba93cc78414e468e3bb02139bc71c0: Log is configured to *not* fsync() on all Append() calls
I20260929 10:36:06.430770 31573 consensus_queue.cc:260] T 81048e496c6e4266861b399223be6a54 P e6581ebffa3246a987710e539ce43fa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6581ebffa3246a987710e539ce43fa3" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 35159 } } peers { permanent_uuid: "04ba93cc78414e468e3bb02139bc71c0" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 44377 } } peers { permanent_uuid: "63f8e03d49954851a2b4b48305144ee5" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 36197 } }
I20260929 10:36:06.431526 31422 heartbeater.cc:503] Master 127.15.82.126:44709 was elected leader, sending a full tablet report...
I20260929 10:36:06.431877 31573 ts_tablet_manager.cc:1428] T 81048e496c6e4266861b399223be6a54 P e6581ebffa3246a987710e539ce43fa3: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260929 10:36:06.432608 31577 tablet_bootstrap.cc:492] T 81048e496c6e4266861b399223be6a54 P 04ba93cc78414e468e3bb02139bc71c0: No bootstrap required, opened a new log
I20260929 10:36:06.432806 31577 ts_tablet_manager.cc:1397] T 81048e496c6e4266861b399223be6a54 P 04ba93cc78414e468e3bb02139bc71c0: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260929 10:36:06.436131 31577 raft_consensus.cc:348] T 81048e496c6e4266861b399223be6a54 P 04ba93cc78414e468e3bb02139bc71c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6581ebffa3246a987710e539ce43fa3" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 35159 } } peers { permanent_uuid: "04ba93cc78414e468e3bb02139bc71c0" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 44377 } } peers { permanent_uuid: "63f8e03d49954851a2b4b48305144ee5" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 36197 } }
I20260929 10:36:06.436417 31577 raft_consensus.cc:374] T 81048e496c6e4266861b399223be6a54 P 04ba93cc78414e468e3bb02139bc71c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:36:06.436478 31577 raft_consensus.cc:729] T 81048e496c6e4266861b399223be6a54 P 04ba93cc78414e468e3bb02139bc71c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04ba93cc78414e468e3bb02139bc71c0, State: Initialized, Role: FOLLOWER
I20260929 10:36:06.437000 31577 consensus_queue.cc:260] T 81048e496c6e4266861b399223be6a54 P 04ba93cc78414e468e3bb02139bc71c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6581ebffa3246a987710e539ce43fa3" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 35159 } } peers { permanent_uuid: "04ba93cc78414e468e3bb02139bc71c0" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 44377 } } peers { permanent_uuid: "63f8e03d49954851a2b4b48305144ee5" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 36197 } }
I20260929 10:36:06.437656 31556 heartbeater.cc:503] Master 127.15.82.126:44709 was elected leader, sending a full tablet report...
I20260929 10:36:06.437963 31577 ts_tablet_manager.cc:1428] T 81048e496c6e4266861b399223be6a54 P 04ba93cc78414e468e3bb02139bc71c0: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20260929 10:36:06.598085 31557 tablet.cc:2369] T 81048e496c6e4266861b399223be6a54 P 04ba93cc78414e468e3bb02139bc71c0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 10:36:06.609589 31423 tablet.cc:2369] T 81048e496c6e4266861b399223be6a54 P e6581ebffa3246a987710e539ce43fa3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 10:36:06.620864 31289 tablet.cc:2369] T 81048e496c6e4266861b399223be6a54 P 63f8e03d49954851a2b4b48305144ee5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 10:36:06.627561 31581 raft_consensus.cc:482] T 81048e496c6e4266861b399223be6a54 P 04ba93cc78414e468e3bb02139bc71c0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:36:06.627750 31581 raft_consensus.cc:504] T 81048e496c6e4266861b399223be6a54 P 04ba93cc78414e468e3bb02139bc71c0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6581ebffa3246a987710e539ce43fa3" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 35159 } } peers { permanent_uuid: "04ba93cc78414e468e3bb02139bc71c0" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 44377 } } peers { permanent_uuid: "63f8e03d49954851a2b4b48305144ee5" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 36197 } }
I20260929 10:36:06.628921 31581 leader_election.cc:288] T 81048e496c6e4266861b399223be6a54 P 04ba93cc78414e468e3bb02139bc71c0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e6581ebffa3246a987710e539ce43fa3 (127.15.82.66:35159), 63f8e03d49954851a2b4b48305144ee5 (127.15.82.65:36197)
I20260929 10:36:06.633275 31558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:06.629151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.66:35159 (local address 127.15.82.67:33071)
0929 10:36:06.629299 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0929 10:36:06.629317 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 10:36:06.629421 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:36:06.629749 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:36:06.629752 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:36:06.629769 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:36:06.629889 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:36:06.629894 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:36:06.630676 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:36:06.630679 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:36:06.631995 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:36:06.632006 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:36:06.632775 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:36:06.632779 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:36:06.632901 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:36:06.632931 (+ 30us) client_negotiation.cc:769] Sending connection context
0929 10:36:06.633005 (+ 74us) client_negotiation.cc:240] Negotiation successful
0929 10:36:06.633135 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57,"mutex_wait_us":27}
I20260929 10:36:06.633317 31568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:06.629258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.67:33071 (local address 127.15.82.66:35159)
0929 10:36:06.629431 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0929 10:36:06.629436 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:36:06.629455 (+ 19us) server_negotiation.cc:407] Connection header received
0929 10:36:06.629598 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:36:06.629601 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:36:06.629645 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:36:06.629737 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:36:06.629998 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:36:06.630563 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:36:06.632135 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:36:06.632653 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:36:06.632746 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:36:06.632787 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0929 10:36:06.633047 (+ 260us) server_negotiation.cc:299] Negotiation successful
0929 10:36:06.633183 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260929 10:36:06.633574 31582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:06.629266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.65:36197 (local address 127.15.82.67:41461)
0929 10:36:06.629591 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0929 10:36:06.629605 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 10:36:06.629717 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:36:06.629988 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:36:06.629990 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:36:06.630005 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:36:06.630123 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:36:06.630129 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:36:06.631027 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:36:06.631030 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:36:06.632237 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:36:06.632245 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:36:06.633122 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:36:06.633125 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:36:06.633228 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:36:06.633252 (+ 24us) client_negotiation.cc:769] Sending connection context
0929 10:36:06.633332 (+ 80us) client_negotiation.cc:240] Negotiation successful
0929 10:36:06.633406 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"mutex_wait_us":25,"thread_start_us":123,"threads_started":1}
I20260929 10:36:06.633726 31572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:06.629469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.67:41461 (local address 127.15.82.65:36197)
0929 10:36:06.629622 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0929 10:36:06.629631 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 10:36:06.629666 (+ 35us) server_negotiation.cc:407] Connection header received
0929 10:36:06.629846 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:36:06.629848 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:36:06.629888 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:36:06.629977 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:36:06.630227 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:36:06.630879 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:36:06.632378 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:36:06.632903 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:36:06.633025 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:36:06.633100 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0929 10:36:06.633503 (+ 403us) server_negotiation.cc:299] Negotiation successful
0929 10:36:06.633593 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260929 10:36:06.633958 31376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81048e496c6e4266861b399223be6a54" candidate_uuid: "04ba93cc78414e468e3bb02139bc71c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6581ebffa3246a987710e539ce43fa3" is_pre_election: true
I20260929 10:36:06.634284 31376 raft_consensus.cc:2454] T 81048e496c6e4266861b399223be6a54 P e6581ebffa3246a987710e539ce43fa3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04ba93cc78414e468e3bb02139bc71c0 in term 0.
I20260929 10:36:06.634570 31242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81048e496c6e4266861b399223be6a54" candidate_uuid: "04ba93cc78414e468e3bb02139bc71c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63f8e03d49954851a2b4b48305144ee5" is_pre_election: true
I20260929 10:36:06.634776 31446 leader_election.cc:302] T 81048e496c6e4266861b399223be6a54 P 04ba93cc78414e468e3bb02139bc71c0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04ba93cc78414e468e3bb02139bc71c0, e6581ebffa3246a987710e539ce43fa3; no voters:
I20260929 10:36:06.634840 31242 raft_consensus.cc:2454] T 81048e496c6e4266861b399223be6a54 P 63f8e03d49954851a2b4b48305144ee5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04ba93cc78414e468e3bb02139bc71c0 in term 0.
I20260929 10:36:06.635090 31581 raft_consensus.cc:2781] T 81048e496c6e4266861b399223be6a54 P 04ba93cc78414e468e3bb02139bc71c0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:36:06.635200 31581 raft_consensus.cc:482] T 81048e496c6e4266861b399223be6a54 P 04ba93cc78414e468e3bb02139bc71c0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:36:06.635252 31581 raft_consensus.cc:3037] T 81048e496c6e4266861b399223be6a54 P 04ba93cc78414e468e3bb02139bc71c0 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:36:06.636286 31581 raft_consensus.cc:504] T 81048e496c6e4266861b399223be6a54 P 04ba93cc78414e468e3bb02139bc71c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6581ebffa3246a987710e539ce43fa3" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 35159 } } peers { permanent_uuid: "04ba93cc78414e468e3bb02139bc71c0" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 44377 } } peers { permanent_uuid: "63f8e03d49954851a2b4b48305144ee5" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 36197 } }
I20260929 10:36:06.636665 31581 leader_election.cc:288] T 81048e496c6e4266861b399223be6a54 P 04ba93cc78414e468e3bb02139bc71c0 [CANDIDATE]: Term 1 election: Requested vote from peers e6581ebffa3246a987710e539ce43fa3 (127.15.82.66:35159), 63f8e03d49954851a2b4b48305144ee5 (127.15.82.65:36197)
I20260929 10:36:06.637015 31242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81048e496c6e4266861b399223be6a54" candidate_uuid: "04ba93cc78414e468e3bb02139bc71c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63f8e03d49954851a2b4b48305144ee5"
I20260929 10:36:06.637089 31376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81048e496c6e4266861b399223be6a54" candidate_uuid: "04ba93cc78414e468e3bb02139bc71c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6581ebffa3246a987710e539ce43fa3"
I20260929 10:36:06.637146 31242 raft_consensus.cc:3037] T 81048e496c6e4266861b399223be6a54 P 63f8e03d49954851a2b4b48305144ee5 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:36:06.637207 31376 raft_consensus.cc:3037] T 81048e496c6e4266861b399223be6a54 P e6581ebffa3246a987710e539ce43fa3 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:36:06.638173 31376 raft_consensus.cc:2454] T 81048e496c6e4266861b399223be6a54 P e6581ebffa3246a987710e539ce43fa3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04ba93cc78414e468e3bb02139bc71c0 in term 1.
I20260929 10:36:06.638183 31242 raft_consensus.cc:2454] T 81048e496c6e4266861b399223be6a54 P 63f8e03d49954851a2b4b48305144ee5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04ba93cc78414e468e3bb02139bc71c0 in term 1.
I20260929 10:36:06.638518 31443 leader_election.cc:302] T 81048e496c6e4266861b399223be6a54 P 04ba93cc78414e468e3bb02139bc71c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04ba93cc78414e468e3bb02139bc71c0, 63f8e03d49954851a2b4b48305144ee5; no voters:
I20260929 10:36:06.638706 31581 raft_consensus.cc:2781] T 81048e496c6e4266861b399223be6a54 P 04ba93cc78414e468e3bb02139bc71c0 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:36:06.638933 31581 raft_consensus.cc:686] T 81048e496c6e4266861b399223be6a54 P 04ba93cc78414e468e3bb02139bc71c0 [term 1 LEADER]: Becoming Leader. State: Replica: 04ba93cc78414e468e3bb02139bc71c0, State: Running, Role: LEADER
I20260929 10:36:06.639285 31581 consensus_queue.cc:237] T 81048e496c6e4266861b399223be6a54 P 04ba93cc78414e468e3bb02139bc71c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6581ebffa3246a987710e539ce43fa3" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 35159 } } peers { permanent_uuid: "04ba93cc78414e468e3bb02139bc71c0" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 44377 } } peers { permanent_uuid: "63f8e03d49954851a2b4b48305144ee5" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 36197 } }
I20260929 10:36:06.642359 31090 catalog_manager.cc:5495] T 81048e496c6e4266861b399223be6a54 P 04ba93cc78414e468e3bb02139bc71c0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 04ba93cc78414e468e3bb02139bc71c0 (127.15.82.67). New cstate: current_term: 1 leader_uuid: "04ba93cc78414e468e3bb02139bc71c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6581ebffa3246a987710e539ce43fa3" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 35159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "04ba93cc78414e468e3bb02139bc71c0" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 44377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "63f8e03d49954851a2b4b48305144ee5" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 36197 } health_report { overall_health: UNKNOWN } } }
I20260929 10:36:06.698336 31570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:06.694197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46054 (local address 127.15.82.67:44377)
0929 10:36:06.694346 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0929 10:36:06.694351 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:36:06.694368 (+ 17us) server_negotiation.cc:407] Connection header received
0929 10:36:06.694542 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:36:06.694544 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:36:06.694602 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:36:06.694738 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:36:06.694979 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:36:06.695585 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:36:06.696450 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:36:06.696701 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:36:06.696817 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:36:06.697237 (+ 420us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:36:06.697843 (+ 606us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:36:06.697991 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0929 10:36:06.698078 (+ 87us) server_negotiation.cc:299] Negotiation successful
0929 10:36:06.698130 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260929 10:36:06.704329 31376 raft_consensus.cc:1261] T 81048e496c6e4266861b399223be6a54 P e6581ebffa3246a987710e539ce43fa3 [term 1 FOLLOWER]: Refusing update from remote peer 04ba93cc78414e468e3bb02139bc71c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:36:06.704329 31242 raft_consensus.cc:1261] T 81048e496c6e4266861b399223be6a54 P 63f8e03d49954851a2b4b48305144ee5 [term 1 FOLLOWER]: Refusing update from remote peer 04ba93cc78414e468e3bb02139bc71c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:36:06.705036 31583 consensus_queue.cc:1035] T 81048e496c6e4266861b399223be6a54 P 04ba93cc78414e468e3bb02139bc71c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6581ebffa3246a987710e539ce43fa3" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 35159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:36:06.705224 31581 consensus_queue.cc:1035] T 81048e496c6e4266861b399223be6a54 P 04ba93cc78414e468e3bb02139bc71c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "63f8e03d49954851a2b4b48305144ee5" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 36197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:36:06.715291 31587 mvcc.cc:205] Tried to move back new op lower bound from 7334617770813833216 to 7334617770556960768. Current Snapshot: MvccSnapshot[applied={T|T < 7334617770813833216}]
I20260929 10:36:06.720714 31090 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59286:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:36:06.720899 31090 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59286:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:36:06.723361 31090 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8fb3386c28a34308a393436c0d1cde01: Sending DeleteTablet for 3 replicas of tablet 81048e496c6e4266861b399223be6a54
I20260929 10:36:06.724205 31490 tablet_service.cc:1514] Processing DeleteTablet for tablet 81048e496c6e4266861b399223be6a54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:36:06 UTC) from {username='kudu', principal='kudu/127.15.82.126@KRBTEST.COM'} at 127.0.0.1:46038
I20260929 10:36:06.724207 31222 tablet_service.cc:1514] Processing DeleteTablet for tablet 81048e496c6e4266861b399223be6a54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:36:06 UTC) from {username='kudu', principal='kudu/127.15.82.126@KRBTEST.COM'} at 127.0.0.1:48264
I20260929 10:36:06.724207 31356 tablet_service.cc:1514] Processing DeleteTablet for tablet 81048e496c6e4266861b399223be6a54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:36:06 UTC) from {username='kudu', principal='kudu/127.15.82.126@KRBTEST.COM'} at 127.0.0.1:53244
I20260929 10:36:06.724743 31598 tablet_replica.cc:330] T 81048e496c6e4266861b399223be6a54 P 63f8e03d49954851a2b4b48305144ee5: stopping tablet replica
I20260929 10:36:06.724957 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 31159
I20260929 10:36:06.724967 31596 tablet_replica.cc:330] T 81048e496c6e4266861b399223be6a54 P 04ba93cc78414e468e3bb02139bc71c0: stopping tablet replica
I20260929 10:36:06.725082 31598 raft_consensus.cc:2229] T 81048e496c6e4266861b399223be6a54 P 63f8e03d49954851a2b4b48305144ee5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:36:06.725284 31596 raft_consensus.cc:2229] T 81048e496c6e4266861b399223be6a54 P 04ba93cc78414e468e3bb02139bc71c0 [term 1 LEADER]: Raft consensus shutting down.
I20260929 10:36:06.725422 31598 raft_consensus.cc:2258] T 81048e496c6e4266861b399223be6a54 P 63f8e03d49954851a2b4b48305144ee5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:36:06.725661 31596 raft_consensus.cc:2258] T 81048e496c6e4266861b399223be6a54 P 04ba93cc78414e468e3bb02139bc71c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:36:06.725720 31597 tablet_replica.cc:330] T 81048e496c6e4266861b399223be6a54 P e6581ebffa3246a987710e539ce43fa3: stopping tablet replica
I20260929 10:36:06.725996 31597 raft_consensus.cc:2229] T 81048e496c6e4266861b399223be6a54 P e6581ebffa3246a987710e539ce43fa3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:36:06.726293 31597 raft_consensus.cc:2258] T 81048e496c6e4266861b399223be6a54 P e6581ebffa3246a987710e539ce43fa3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:36:06.727870 31597 ts_tablet_manager.cc:1905] T 81048e496c6e4266861b399223be6a54 P e6581ebffa3246a987710e539ce43fa3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:36:06.728302 31596 ts_tablet_manager.cc:1905] T 81048e496c6e4266861b399223be6a54 P 04ba93cc78414e468e3bb02139bc71c0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:36:06.730890 31597 ts_tablet_manager.cc:1918] T 81048e496c6e4266861b399223be6a54 P e6581ebffa3246a987710e539ce43fa3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:36:06.731005 31597 log.cc:1196] T 81048e496c6e4266861b399223be6a54 P e6581ebffa3246a987710e539ce43fa3: Deleting WAL directory at /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-1/wal/wals/81048e496c6e4266861b399223be6a54
I20260929 10:36:06.731352 31597 ts_tablet_manager.cc:1939] T 81048e496c6e4266861b399223be6a54 P e6581ebffa3246a987710e539ce43fa3: Deleting consensus metadata
I20260929 10:36:06.732013 31596 ts_tablet_manager.cc:1918] T 81048e496c6e4266861b399223be6a54 P 04ba93cc78414e468e3bb02139bc71c0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:36:06.732131 31596 log.cc:1196] T 81048e496c6e4266861b399223be6a54 P 04ba93cc78414e468e3bb02139bc71c0: Deleting WAL directory at /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-2/wal/wals/81048e496c6e4266861b399223be6a54
I20260929 10:36:06.732471 31596 ts_tablet_manager.cc:1939] T 81048e496c6e4266861b399223be6a54 P 04ba93cc78414e468e3bb02139bc71c0: Deleting consensus metadata
I20260929 10:36:06.732764 31084 catalog_manager.cc:4841] TS e6581ebffa3246a987710e539ce43fa3 (127.15.82.66:35159): tablet 81048e496c6e4266861b399223be6a54 (table test-table [id=bb7e6acae25f4b819b42b2820d6424b2]) successfully deleted
I20260929 10:36:06.733747 31084 catalog_manager.cc:4841] TS 04ba93cc78414e468e3bb02139bc71c0 (127.15.82.67:44377): tablet 81048e496c6e4266861b399223be6a54 (table test-table [id=bb7e6acae25f4b819b42b2820d6424b2]) successfully deleted
W20260929 10:36:06.735100 31081 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.82.65:36197 (error 108)
I20260929 10:36:06.735304 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 31293
I20260929 10:36:06.741288 31567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:06.740963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.65:36197 (local address 127.0.0.1:48274)
0929 10:36:06.741117 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0929 10:36:06.741195 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.65:36197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260929 10:36:06.741557 31081 catalog_manager.cc:4584] TS 63f8e03d49954851a2b4b48305144ee5 (127.15.82.65:36197): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 81048e496c6e4266861b399223be6a54: Network error: Client connection negotiation failed: client connection to 127.15.82.65:36197: connect: Connection refused (error 111)
I20260929 10:36:06.741628 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 31427
I20260929 10:36:06.747290 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 31065
I20260929 10:36:26.754483 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:44709
--webserver_interface=127.15.82.126
--webserver_port=36519
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab
--principal=kudu/127.15.82.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:34999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.82.126:44709
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:36:26.864760 31600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:36:26.865036 31600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:36:26.865077 31600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:36:26.869062 31600 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260929 10:36:26.869112 31600 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260929 10:36:26.869127 31600 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260929 10:36:26.869141 31600 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:36:26.869160 31600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:36:26.869217 31600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:36:26.869249 31600 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:36:26.869271 31600 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:36:26.874534 31600 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:34999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.15.82.126:44709
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab
--principal=kudu/127.15.82.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:44709
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.82.126
--webserver_port=36519
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31600
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:36:26.875631 31600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:36:26.876735 31600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:36:26.890194 31607 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:36:26.890327 31600 server_base.cc:1034] running on GCE node
W20260929 10:36:26.890262 31609 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:36:26.890280 31606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:36:26.891266 31600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:36:26.892457 31600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:36:26.893656 31600 hybrid_clock.cc:648] HybridClock initialized: now 1790678186893631 us; error 38 us; skew 500 ppm
Sep 29 10:36:26 dist-test-slave-vg89 krb5kdc[31052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678186, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:36:26.902043 31600 init.cc:377] Logged in from keytab as kudu/127.15.82.126@KRBTEST.COM (short username kudu)
I20260929 10:36:26.903067 31600 webserver.cc:460] Webserver started at http://127.15.82.126:36519/ using document root <none> and password file <none>
I20260929 10:36:26.903671 31600 fs_manager.cc:359] Metadata directory not provided
I20260929 10:36:26.903770 31600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:36:26.907917 31600 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260929 10:36:26.910157 31616 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:36:26.911445 31600 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 10:36:26.911593 31600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/master-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/master-0/wal
uuid: "8fb3386c28a34308a393436c0d1cde01"
format_stamp: "Formatted at 2026-09-29 10:36:05 on dist-test-slave-vg89"
server_key: "1542023ca7ac7ab95c4e1ede5b9e973f"
server_key_iv: "939e30e658d7c6464f336ce80cee90ae"
server_key_version: "encryptionkey@0"
I20260929 10:36:26.912196 31600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:36:26.927179 31600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:36:26.930083 31600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:36:26.930310 31600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:36:26.937445 31600 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.126:44709
I20260929 10:36:26.937726 31668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.126:44709 every 8 connection(s)
I20260929 10:36:26.938581 31600 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
I20260929 10:36:26.941418 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 31600
I20260929 10:36:26.941895 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.65:36197
--local_ip_for_outbound_sockets=127.15.82.65
--tserver_master_addrs=127.15.82.126:44709
--webserver_port=36785
--webserver_interface=127.15.82.65
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab
--principal=kudu/127.15.82.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:34999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 10:36:26.943006 31669 sys_catalog.cc:263] Verifying existing consensus state
I20260929 10:36:26.944628 31669 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fb3386c28a34308a393436c0d1cde01: Bootstrap starting.
I20260929 10:36:26.961429 31669 log.cc:824] T 00000000000000000000000000000000 P 8fb3386c28a34308a393436c0d1cde01: Log is configured to *not* fsync() on all Append() calls
I20260929 10:36:26.969772 31669 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fb3386c28a34308a393436c0d1cde01: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 10:36:26.970160 31669 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fb3386c28a34308a393436c0d1cde01: Bootstrap complete.
I20260929 10:36:26.974167 31669 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8fb3386c28a34308a393436c0d1cde01 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fb3386c28a34308a393436c0d1cde01" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 44709 } }
I20260929 10:36:26.974644 31669 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8fb3386c28a34308a393436c0d1cde01 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fb3386c28a34308a393436c0d1cde01, State: Initialized, Role: FOLLOWER
I20260929 10:36:26.975179 31669 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8fb3386c28a34308a393436c0d1cde01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fb3386c28a34308a393436c0d1cde01" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 44709 } }
I20260929 10:36:26.975324 31669 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8fb3386c28a34308a393436c0d1cde01 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:36:26.975405 31669 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8fb3386c28a34308a393436c0d1cde01 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:36:26.975546 31669 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8fb3386c28a34308a393436c0d1cde01 [term 1 FOLLOWER]: Advancing to term 2
I20260929 10:36:26.977437 31669 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8fb3386c28a34308a393436c0d1cde01 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fb3386c28a34308a393436c0d1cde01" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 44709 } }
I20260929 10:36:26.977761 31669 leader_election.cc:302] T 00000000000000000000000000000000 P 8fb3386c28a34308a393436c0d1cde01 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8fb3386c28a34308a393436c0d1cde01; no voters:
I20260929 10:36:26.978087 31669 leader_election.cc:288] T 00000000000000000000000000000000 P 8fb3386c28a34308a393436c0d1cde01 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260929 10:36:26.978166 31673 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8fb3386c28a34308a393436c0d1cde01 [term 2 FOLLOWER]: Leader election won for term 2
I20260929 10:36:26.978422 31673 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8fb3386c28a34308a393436c0d1cde01 [term 2 LEADER]: Becoming Leader. State: Replica: 8fb3386c28a34308a393436c0d1cde01, State: Running, Role: LEADER
I20260929 10:36:26.978725 31673 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8fb3386c28a34308a393436c0d1cde01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fb3386c28a34308a393436c0d1cde01" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 44709 } }
I20260929 10:36:26.979354 31669 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8fb3386c28a34308a393436c0d1cde01 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:36:26.979636 31673 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fb3386c28a34308a393436c0d1cde01 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "8fb3386c28a34308a393436c0d1cde01" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fb3386c28a34308a393436c0d1cde01" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 44709 } } }
I20260929 10:36:26.979750 31673 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fb3386c28a34308a393436c0d1cde01 [sys.catalog]: This master's current role is: LEADER
I20260929 10:36:26.980156 31674 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fb3386c28a34308a393436c0d1cde01 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8fb3386c28a34308a393436c0d1cde01. Latest consensus state: current_term: 2 leader_uuid: "8fb3386c28a34308a393436c0d1cde01" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fb3386c28a34308a393436c0d1cde01" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 44709 } } }
I20260929 10:36:26.980336 31674 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fb3386c28a34308a393436c0d1cde01 [sys.catalog]: This master's current role is: LEADER
I20260929 10:36:26.984424 31689 catalog_manager.cc:1259] Loaded cluster ID: b3abbf64430d41ac8eec98adc906db15
I20260929 10:36:26.984534 31689 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8fb3386c28a34308a393436c0d1cde01: loading cluster ID for follower catalog manager: success
I20260929 10:36:26.987354 31689 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8fb3386c28a34308a393436c0d1cde01: acquiring CA information for follower catalog manager: success
I20260929 10:36:26.988889 31689 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8fb3386c28a34308a393436c0d1cde01: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 10:36:26.989115 31677 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:36:26.991487 31677 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:36:26.991953 31677 catalog_manager.cc:1259] Loaded cluster ID: b3abbf64430d41ac8eec98adc906db15
I20260929 10:36:26.992010 31677 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:36:26.992632 31677 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:36:26.993104 31677 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8fb3386c28a34308a393436c0d1cde01: Loaded TSK: 0
I20260929 10:36:27.006479 31677 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8fb3386c28a34308a393436c0d1cde01: Generated new TSK 1
I20260929 10:36:27.006719 31677 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260929 10:36:27.058034 31671 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:36:27.058310 31671 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:36:27.058429 31671 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:36:27.062462 31671 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:36:27.062557 31671 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:36:27.062651 31671 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.65
I20260929 10:36:27.068032 31671 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:34999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab
--principal=kudu/127.15.82.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.65:36197
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.82.65
--webserver_port=36785
--tserver_master_addrs=127.15.82.126:44709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31671
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.65
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:36:27.069101 31671 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:36:27.070137 31671 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:36:27.080943 31696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:36:27.080942 31697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:36:27.080976 31699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:36:27.081524 31671 server_base.cc:1034] running on GCE node
I20260929 10:36:27.081960 31671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:36:27.082731 31671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:36:27.083917 31671 hybrid_clock.cc:648] HybridClock initialized: now 1790678187083889 us; error 45 us; skew 500 ppm
Sep 29 10:36:27 dist-test-slave-vg89 krb5kdc[31052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678187, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:36:27.087069 31671 init.cc:377] Logged in from keytab as kudu/127.15.82.65@KRBTEST.COM (short username kudu)
I20260929 10:36:27.088156 31671 webserver.cc:460] Webserver started at http://127.15.82.65:36785/ using document root <none> and password file <none>
I20260929 10:36:27.088737 31671 fs_manager.cc:359] Metadata directory not provided
I20260929 10:36:27.088845 31671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:36:27.092995 31671 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260929 10:36:27.095240 31706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:36:27.096619 31671 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260929 10:36:27.096738 31671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-0/wal
uuid: "63f8e03d49954851a2b4b48305144ee5"
format_stamp: "Formatted at 2026-09-29 10:36:05 on dist-test-slave-vg89"
server_key: "3572e645b2fd051b89fa5a6e2866f4ac"
server_key_iv: "3a2d499a1385e60427468602d7a17cdc"
server_key_version: "encryptionkey@0"
I20260929 10:36:27.097285 31671 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:36:27.108042 31671 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:36:27.111235 31671 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:36:27.111466 31671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:36:27.112293 31671 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:36:27.113495 31713 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260929 10:36:27.117766 31671 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260929 10:36:27.117830 31671 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260929 10:36:27.117900 31671 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260929 10:36:27.119692 31671 ts_tablet_manager.cc:610] Registered 1 tablets
I20260929 10:36:27.119750 31671 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260929 10:36:27.120010 31713 tablet_bootstrap.cc:492] T 81048e496c6e4266861b399223be6a54 P 63f8e03d49954851a2b4b48305144ee5: Bootstrap starting.
I20260929 10:36:27.130756 31671 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.65:36197
I20260929 10:36:27.132081 31671 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
I20260929 10:36:27.134299 31820 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.65:36197 every 8 connection(s)
I20260929 10:36:27.138783 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 31671
I20260929 10:36:27.139360 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.66:35159
--local_ip_for_outbound_sockets=127.15.82.66
--tserver_master_addrs=127.15.82.126:44709
--webserver_port=33533
--webserver_interface=127.15.82.66
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab
--principal=kudu/127.15.82.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:34999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 10:36:27.141397 31713 log.cc:824] T 81048e496c6e4266861b399223be6a54 P 63f8e03d49954851a2b4b48305144ee5: Log is configured to *not* fsync() on all Append() calls
Sep 29 10:36:27 dist-test-slave-vg89 krb5kdc[31052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678187, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.65@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:36:27.149402 31713 tablet_bootstrap.cc:492] T 81048e496c6e4266861b399223be6a54 P 63f8e03d49954851a2b4b48305144ee5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 10:36:27.149863 31713 tablet_bootstrap.cc:492] T 81048e496c6e4266861b399223be6a54 P 63f8e03d49954851a2b4b48305144ee5: Bootstrap complete.
I20260929 10:36:27.151154 31824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:27.134601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:44709 (local address 127.15.82.65:39665)
0929 10:36:27.135221 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0929 10:36:27.135261 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0929 10:36:27.136078 (+ 817us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:36:27.137163 (+ 1085us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:36:27.137170 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:36:27.137603 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:36:27.138055 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:36:27.138068 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:36:27.139706 (+ 1638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:36:27.139709 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:36:27.140583 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:36:27.140592 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:36:27.141636 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:36:27.141640 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:36:27.141854 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:36:27.142546 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:36:27.142572 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:36:27.144500 (+ 1928us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:36:27.147461 (+ 2961us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:36:27.147466 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:36:27.147479 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:36:27.147815 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:36:27.148273 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:36:27.148276 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:36:27.148277 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:36:27.148451 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:36:27.149254 (+ 803us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:36:27.149259 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:36:27.149543 (+ 284us) client_negotiation.cc:769] Sending connection context
0929 10:36:27.149756 (+ 213us) client_negotiation.cc:240] Negotiation successful
0929 10:36:27.150098 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":145,"threads_started":1}
I20260929 10:36:27.151224 31713 ts_tablet_manager.cc:1397] T 81048e496c6e4266861b399223be6a54 P 63f8e03d49954851a2b4b48305144ee5: Time spent bootstrapping tablet: real 0.031s user 0.021s sys 0.008s
I20260929 10:36:27.151723 31823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:27.134600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.65:39665 (local address 127.15.82.126:44709)
0929 10:36:27.134976 (+ 376us) server_negotiation.cc:206] Beginning negotiation
0929 10:36:27.134991 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 10:36:27.135382 (+ 391us) server_negotiation.cc:407] Connection header received
0929 10:36:27.136481 (+ 1099us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:36:27.136491 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:36:27.136773 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:36:27.137110 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:36:27.138230 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:36:27.139369 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:36:27.140712 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:36:27.141431 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:36:27.141605 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:36:27.144736 (+ 3131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:36:27.144746 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:36:27.144768 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:36:27.144813 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:36:27.147313 (+ 2500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:36:27.147956 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:36:27.147973 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:36:27.147982 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:36:27.148159 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:36:27.148558 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:36:27.148563 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:36:27.148568 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:36:27.149116 (+ 548us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:36:27.149270 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0929 10:36:27.149986 (+ 716us) server_negotiation.cc:299] Negotiation successful
0929 10:36:27.150332 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":103,"threads_started":1}
I20260929 10:36:27.154107 31821 heartbeater.cc:347] Connected to a master server at 127.15.82.126:44709
I20260929 10:36:27.153885 31713 raft_consensus.cc:348] T 81048e496c6e4266861b399223be6a54 P 63f8e03d49954851a2b4b48305144ee5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6581ebffa3246a987710e539ce43fa3" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 35159 } } peers { permanent_uuid: "04ba93cc78414e468e3bb02139bc71c0" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 44377 } } peers { permanent_uuid: "63f8e03d49954851a2b4b48305144ee5" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 36197 } }
I20260929 10:36:27.154351 31821 heartbeater.cc:464] Registering TS with master...
I20260929 10:36:27.154460 31713 raft_consensus.cc:729] T 81048e496c6e4266861b399223be6a54 P 63f8e03d49954851a2b4b48305144ee5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63f8e03d49954851a2b4b48305144ee5, State: Initialized, Role: FOLLOWER
I20260929 10:36:27.154945 31821 heartbeater.cc:511] Master 127.15.82.126:44709 requested a full tablet report, sending...
I20260929 10:36:27.154949 31713 consensus_queue.cc:260] T 81048e496c6e4266861b399223be6a54 P 63f8e03d49954851a2b4b48305144ee5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6581ebffa3246a987710e539ce43fa3" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 35159 } } peers { permanent_uuid: "04ba93cc78414e468e3bb02139bc71c0" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 44377 } } peers { permanent_uuid: "63f8e03d49954851a2b4b48305144ee5" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 36197 } }
I20260929 10:36:27.156019 31713 ts_tablet_manager.cc:1428] T 81048e496c6e4266861b399223be6a54 P 63f8e03d49954851a2b4b48305144ee5: Time spent starting tablet: real 0.005s user 0.000s sys 0.005s
I20260929 10:36:27.157510 31633 ts_manager.cc:194] Registered new tserver with Master: 63f8e03d49954851a2b4b48305144ee5 (127.15.82.65:36197)
I20260929 10:36:27.161173 31633 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.82.65@KRBTEST.COM'} at 127.15.82.65:39665
I20260929 10:36:27.163951 31821 heartbeater.cc:503] Master 127.15.82.126:44709 was elected leader, sending a full tablet report...
Sep 29 10:36:27 dist-test-slave-vg89 krb5kdc[31052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678186, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.126@KRBTEST.COM for kudu/127.15.82.65@KRBTEST.COM
I20260929 10:36:27.172108 31828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:27.160282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.65:36197 (local address 127.0.0.1:34220)
0929 10:36:27.160690 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0929 10:36:27.160737 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0929 10:36:27.160857 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:36:27.161499 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:36:27.161506 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:36:27.161940 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:36:27.162070 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:36:27.162079 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:36:27.165109 (+ 3030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:36:27.165116 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:36:27.165902 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:36:27.165912 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:36:27.166532 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:36:27.166536 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:36:27.166654 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:36:27.167224 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:36:27.167236 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:36:27.168549 (+ 1313us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:36:27.170344 (+ 1795us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:36:27.170355 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:36:27.170361 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:36:27.170624 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:36:27.170927 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:36:27.170930 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:36:27.170932 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:36:27.171014 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:36:27.171461 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:36:27.171467 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:36:27.171589 (+ 122us) client_negotiation.cc:769] Sending connection context
0929 10:36:27.171725 (+ 136us) client_negotiation.cc:240] Negotiation successful
0929 10:36:27.171928 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":119,"threads_started":1}
I20260929 10:36:27.172108 31829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:27.160347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34220 (local address 127.15.82.65:36197)
0929 10:36:27.161051 (+ 704us) server_negotiation.cc:206] Beginning negotiation
0929 10:36:27.161061 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 10:36:27.161085 (+ 24us) server_negotiation.cc:407] Connection header received
0929 10:36:27.161151 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:36:27.161156 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:36:27.161345 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:36:27.161475 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:36:27.162196 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:36:27.164992 (+ 2796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:36:27.166018 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:36:27.166362 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:36:27.166532 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:36:27.168686 (+ 2154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:36:27.168695 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:36:27.168707 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:36:27.168737 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:36:27.170229 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:36:27.170739 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:36:27.170745 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:36:27.170747 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:36:27.170829 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:36:27.171105 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:36:27.171109 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:36:27.171111 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:36:27.171339 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:36:27.171484 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0929 10:36:27.171803 (+ 319us) server_negotiation.cc:299] Negotiation successful
0929 10:36:27.171962 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":597,"thread_start_us":107,"threads_started":1}
I20260929 10:36:27.173406 31755 tablet_service.cc:1514] Processing DeleteTablet for tablet 81048e496c6e4266861b399223be6a54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:36:06 UTC) from {username='kudu', principal='kudu/127.15.82.126@KRBTEST.COM'} at 127.0.0.1:34220
I20260929 10:36:27.174263 31830 tablet_replica.cc:330] stopping tablet replica
I20260929 10:36:27.174631 31830 raft_consensus.cc:2229] T 81048e496c6e4266861b399223be6a54 P 63f8e03d49954851a2b4b48305144ee5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:36:27.174886 31830 raft_consensus.cc:2258] T 81048e496c6e4266861b399223be6a54 P 63f8e03d49954851a2b4b48305144ee5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:36:27.175981 31830 ts_tablet_manager.cc:1905] T 81048e496c6e4266861b399223be6a54 P 63f8e03d49954851a2b4b48305144ee5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:36:27.180608 31830 ts_tablet_manager.cc:1918] T 81048e496c6e4266861b399223be6a54 P 63f8e03d49954851a2b4b48305144ee5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:36:27.180707 31830 log.cc:1196] T 81048e496c6e4266861b399223be6a54 P 63f8e03d49954851a2b4b48305144ee5: Deleting WAL directory at /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-0/wal/wals/81048e496c6e4266861b399223be6a54
I20260929 10:36:27.180997 31830 ts_tablet_manager.cc:1939] T 81048e496c6e4266861b399223be6a54 P 63f8e03d49954851a2b4b48305144ee5: Deleting consensus metadata
I20260929 10:36:27.183110 31617 catalog_manager.cc:4841] TS 63f8e03d49954851a2b4b48305144ee5 (127.15.82.65:36197): tablet 81048e496c6e4266861b399223be6a54 (table test-table [id=bb7e6acae25f4b819b42b2820d6424b2]) successfully deleted
W20260929 10:36:27.260522 31825 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:36:27.260779 31825 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:36:27.260830 31825 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:36:27.264711 31825 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:36:27.264765 31825 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:36:27.264827 31825 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.66
I20260929 10:36:27.269951 31825 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:34999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab
--principal=kudu/127.15.82.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.66:35159
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.82.66
--webserver_port=33533
--tserver_master_addrs=127.15.82.126:44709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31825
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.66
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:36:27.271339 31825 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:36:27.272500 31825 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:36:27.282539 31835 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:36:27.282573 31836 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:36:27.282539 31838 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:36:27.282931 31825 server_base.cc:1034] running on GCE node
I20260929 10:36:27.283353 31825 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:36:27.284039 31825 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:36:27.285252 31825 hybrid_clock.cc:648] HybridClock initialized: now 1790678187285224 us; error 37 us; skew 500 ppm
Sep 29 10:36:27 dist-test-slave-vg89 krb5kdc[31052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678187, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:36:27.288081 31825 init.cc:377] Logged in from keytab as kudu/127.15.82.66@KRBTEST.COM (short username kudu)
I20260929 10:36:27.288936 31825 webserver.cc:460] Webserver started at http://127.15.82.66:33533/ using document root <none> and password file <none>
I20260929 10:36:27.289532 31825 fs_manager.cc:359] Metadata directory not provided
I20260929 10:36:27.289640 31825 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:36:27.293586 31825 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260929 10:36:27.295682 31845 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:36:27.296969 31825 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260929 10:36:27.297127 31825 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-1/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-1/wal
uuid: "e6581ebffa3246a987710e539ce43fa3"
format_stamp: "Formatted at 2026-09-29 10:36:06 on dist-test-slave-vg89"
server_key: "0bfe256a079924566e473b8a3bd241f8"
server_key_iv: "8d5aeb5922cc2793b1b2e4164c640eff"
server_key_version: "encryptionkey@0"
I20260929 10:36:27.297673 31825 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:36:27.318902 31825 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:36:27.321738 31825 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:36:27.321942 31825 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:36:27.322616 31825 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:36:27.323455 31825 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:36:27.323501 31825 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:36:27.323566 31825 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:36:27.323601 31825 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:36:27.332463 31825 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.66:35159
I20260929 10:36:27.332484 31958 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.66:35159 every 8 connection(s)
I20260929 10:36:27.333456 31825 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
I20260929 10:36:27.337384 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 31825
I20260929 10:36:27.337982 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.67:44377
--local_ip_for_outbound_sockets=127.15.82.67
--tserver_master_addrs=127.15.82.126:44709
--webserver_port=36695
--webserver_interface=127.15.82.67
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab
--principal=kudu/127.15.82.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:34999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 10:36:27 dist-test-slave-vg89 krb5kdc[31052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678187, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.66@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:36:27.349093 31823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:27.335297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:50349 (local address 127.15.82.126:44709)
0929 10:36:27.335437 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0929 10:36:27.335441 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:36:27.336448 (+ 1007us) server_negotiation.cc:407] Connection header received
0929 10:36:27.337515 (+ 1067us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:36:27.337519 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:36:27.337582 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:36:27.337659 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:36:27.339406 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:36:27.339934 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:36:27.341597 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:36:27.342045 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:36:27.342176 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:36:27.344942 (+ 2766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:36:27.344949 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:36:27.344954 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:36:27.345000 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:36:27.346831 (+ 1831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:36:27.347346 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:36:27.347353 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:36:27.347355 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:36:27.347453 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:36:27.347781 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:36:27.347786 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:36:27.347791 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:36:27.348078 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:36:27.348198 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0929 10:36:27.348677 (+ 479us) server_negotiation.cc:299] Negotiation successful
0929 10:36:27.348839 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260929 10:36:27.349778 31961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:27.335678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:44709 (local address 127.15.82.66:50349)
0929 10:36:27.336295 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0929 10:36:27.336335 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0929 10:36:27.337220 (+ 885us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:36:27.337885 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:36:27.337897 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:36:27.338456 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:36:27.339190 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:36:27.339214 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:36:27.340070 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:36:27.340076 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:36:27.341449 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:36:27.341459 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:36:27.342198 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:36:27.342200 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:36:27.342343 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:36:27.343043 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:36:27.343058 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:36:27.344748 (+ 1690us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:36:27.346970 (+ 2222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:36:27.346974 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:36:27.346987 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:36:27.347241 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:36:27.347552 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:36:27.347554 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:36:27.347556 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:36:27.347684 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:36:27.348188 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:36:27.348193 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:36:27.348431 (+ 238us) client_negotiation.cc:769] Sending connection context
0929 10:36:27.348587 (+ 156us) client_negotiation.cc:240] Negotiation successful
0929 10:36:27.348858 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":100,"threads_started":1}
I20260929 10:36:27.351243 31959 heartbeater.cc:347] Connected to a master server at 127.15.82.126:44709
I20260929 10:36:27.351514 31959 heartbeater.cc:464] Registering TS with master...
I20260929 10:36:27.352012 31959 heartbeater.cc:511] Master 127.15.82.126:44709 requested a full tablet report, sending...
I20260929 10:36:27.353089 31633 ts_manager.cc:194] Registered new tserver with Master: e6581ebffa3246a987710e539ce43fa3 (127.15.82.66:35159)
I20260929 10:36:27.353678 31633 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.82.66@KRBTEST.COM'} at 127.15.82.66:50349
W20260929 10:36:27.464310 31962 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:36:27.464649 31962 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:36:27.464736 31962 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:36:27.468784 31962 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:36:27.468889 31962 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:36:27.468988 31962 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.67
I20260929 10:36:27.474247 31962 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:34999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab
--principal=kudu/127.15.82.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.67:44377
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.82.67
--webserver_port=36695
--tserver_master_addrs=127.15.82.126:44709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31962
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.67
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:36:27.475390 31962 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:36:27.476649 31962 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:36:27.486758 31970 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:36:27.486737 31968 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:36:27.486737 31967 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:36:27.487108 31962 server_base.cc:1034] running on GCE node
I20260929 10:36:27.487525 31962 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:36:27.488152 31962 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:36:27.489331 31962 hybrid_clock.cc:648] HybridClock initialized: now 1790678187489294 us; error 47 us; skew 500 ppm
Sep 29 10:36:27 dist-test-slave-vg89 krb5kdc[31052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678187, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:36:27.492097 31962 init.cc:377] Logged in from keytab as kudu/127.15.82.67@KRBTEST.COM (short username kudu)
I20260929 10:36:27.492976 31962 webserver.cc:460] Webserver started at http://127.15.82.67:36695/ using document root <none> and password file <none>
I20260929 10:36:27.493547 31962 fs_manager.cc:359] Metadata directory not provided
I20260929 10:36:27.493618 31962 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:36:27.497622 31962 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260929 10:36:27.499828 31977 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:36:27.501101 31962 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260929 10:36:27.501225 31962 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-2/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-2/wal
uuid: "04ba93cc78414e468e3bb02139bc71c0"
format_stamp: "Formatted at 2026-09-29 10:36:06 on dist-test-slave-vg89"
server_key: "91fdf326a1efd5e729607d530e8a9406"
server_key_iv: "50bd17be0b3dda8ae4f58dabbeaebb19"
server_key_version: "encryptionkey@0"
I20260929 10:36:27.501869 31962 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:36:27.519424 31962 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:36:27.522246 31962 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:36:27.522507 31962 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:36:27.523129 31962 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:36:27.524025 31962 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:36:27.524071 31962 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:36:27.524139 31962 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:36:27.524179 31962 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:36:27.532857 31962 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.67:44377
I20260929 10:36:27.532884 32090 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.67:44377 every 8 connection(s)
I20260929 10:36:27.533870 31962 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
I20260929 10:36:27.536031 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 31962
Sep 29 10:36:27 dist-test-slave-vg89 krb5kdc[31052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678187, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.67@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:36:27.544891 32094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:27.536675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55176 (local address 127.15.82.126:44709)
0929 10:36:27.536938 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0929 10:36:27.536943 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:36:27.537163 (+ 220us) server_negotiation.cc:407] Connection header received
0929 10:36:27.537333 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:36:27.537338 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:36:27.537394 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:36:27.537500 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:36:27.538239 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:36:27.538893 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:36:27.539787 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:36:27.540044 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:36:27.540157 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:36:27.541580 (+ 1423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:36:27.541584 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:36:27.541587 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:36:27.541612 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:36:27.543176 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:36:27.543581 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:36:27.543586 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:36:27.543588 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:36:27.543690 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:36:27.543984 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:36:27.543986 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:36:27.543987 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:36:27.544151 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:36:27.544375 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0929 10:36:27.544543 (+ 168us) server_negotiation.cc:299] Negotiation successful
0929 10:36:27.544674 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":89,"threads_started":1}
I20260929 10:36:27.548202 31823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:27.535685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.67:45583 (local address 127.15.82.126:44709)
0929 10:36:27.535867 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0929 10:36:27.535873 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 10:36:27.537206 (+ 1333us) server_negotiation.cc:407] Connection header received
0929 10:36:27.538091 (+ 885us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:36:27.538094 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:36:27.538139 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:36:27.538212 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:36:27.539491 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:36:27.539986 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:36:27.541120 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:36:27.541338 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:36:27.541456 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:36:27.544254 (+ 2798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:36:27.544257 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:36:27.544260 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:36:27.544280 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:36:27.545952 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:36:27.546974 (+ 1022us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:36:27.546977 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:36:27.546978 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:36:27.547049 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:36:27.547344 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:36:27.547346 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:36:27.547348 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:36:27.547499 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:36:27.547568 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0929 10:36:27.548014 (+ 446us) server_negotiation.cc:299] Negotiation successful
0929 10:36:27.548096 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260929 10:36:27.549077 32093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:27.536069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:44709 (local address 127.15.82.67:45583)
0929 10:36:27.537050 (+ 981us) negotiation.cc:107] Waiting for socket to connect
0929 10:36:27.537106 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0929 10:36:27.537884 (+ 778us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:36:27.538450 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:36:27.538461 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:36:27.538839 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:36:27.539350 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:36:27.539363 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:36:27.540110 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:36:27.540114 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:36:27.540996 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:36:27.541004 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:36:27.541495 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:36:27.541500 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:36:27.541677 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:36:27.542331 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:36:27.542357 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:36:27.544105 (+ 1748us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:36:27.546508 (+ 2403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:36:27.546514 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:36:27.546530 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:36:27.546879 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:36:27.547131 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:36:27.547134 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:36:27.547135 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:36:27.547262 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:36:27.547580 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:36:27.547585 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:36:27.547811 (+ 226us) client_negotiation.cc:769] Sending connection context
0929 10:36:27.547957 (+ 146us) client_negotiation.cc:240] Negotiation successful
0929 10:36:27.548203 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":770,"thread_start_us":145,"threads_started":1}
I20260929 10:36:27.550102 32091 heartbeater.cc:347] Connected to a master server at 127.15.82.126:44709
I20260929 10:36:27.550349 32091 heartbeater.cc:464] Registering TS with master...
I20260929 10:36:27.550863 32091 heartbeater.cc:511] Master 127.15.82.126:44709 requested a full tablet report, sending...
I20260929 10:36:27.551824 31633 ts_manager.cc:194] Registered new tserver with Master: 04ba93cc78414e468e3bb02139bc71c0 (127.15.82.67:44377)
I20260929 10:36:27.552366 31633 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.82.67@KRBTEST.COM'} at 127.15.82.67:45583
I20260929 10:36:27.558849 15689 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 10:36:27.565790 31823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:27.559480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55184 (local address 127.15.82.126:44709)
0929 10:36:27.559610 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0929 10:36:27.559617 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 10:36:27.561884 (+ 2267us) server_negotiation.cc:407] Connection header received
0929 10:36:27.562064 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:36:27.562069 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:36:27.562190 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:36:27.562303 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:36:27.562618 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:36:27.563391 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:36:27.564494 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:36:27.564829 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:36:27.564936 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:36:27.565265 (+ 329us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:36:27.565421 (+ 156us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0929 10:36:27.565665 (+ 244us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55184: token expired
Metrics: {"server-negotiator.queue_time_us":66}
W20260929 10:36:27.565845 31823 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55184: token expired
W20260929 10:36:27.571966 31562 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.15.82.126:44709)
I20260929 10:36:27.579392 31823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:27.572380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55186 (local address 127.15.82.126:44709)
0929 10:36:27.572591 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0929 10:36:27.572596 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:36:27.572611 (+ 15us) server_negotiation.cc:407] Connection header received
0929 10:36:27.572659 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:36:27.572663 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:36:27.572710 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:36:27.572794 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:36:27.573479 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:36:27.573952 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:36:27.574789 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:36:27.575022 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:36:27.575137 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:36:27.576413 (+ 1276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:36:27.576416 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:36:27.576418 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:36:27.576440 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:36:27.577902 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:36:27.578260 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:36:27.578263 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:36:27.578264 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:36:27.578335 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:36:27.578623 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:36:27.578625 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:36:27.578626 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:36:27.578790 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:36:27.578899 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0929 10:36:27.579070 (+ 171us) server_negotiation.cc:299] Negotiation successful
0929 10:36:27.579224 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260929 10:36:27.582139 31633 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55186:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 10:36:27.585600 31633 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:36:27.593600 31755 tablet_service.cc:1467] Processing CreateTablet for tablet e757f25757a940fba2a32c51bc0ac3bf (DEFAULT_TABLE table=test-table [id=485a152dedb8432d96e5766c2d7347d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:36:27.594269 31755 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e757f25757a940fba2a32c51bc0ac3bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:36:27.597198 31713 tablet_bootstrap.cc:492] T e757f25757a940fba2a32c51bc0ac3bf P 63f8e03d49954851a2b4b48305144ee5: Bootstrap starting.
I20260929 10:36:27.598136 31713 tablet_bootstrap.cc:654] T e757f25757a940fba2a32c51bc0ac3bf P 63f8e03d49954851a2b4b48305144ee5: Neither blocks nor log segments found. Creating new log.
I20260929 10:36:27.598249 31828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:27.592568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.66:35159 (local address 127.0.0.1:58240)
0929 10:36:27.592675 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0929 10:36:27.592691 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 10:36:27.592774 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:36:27.594117 (+ 1343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:36:27.594120 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:36:27.594138 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:36:27.594248 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:36:27.594255 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:36:27.595778 (+ 1523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:36:27.595784 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:36:27.596598 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:36:27.596607 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:36:27.597765 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:36:27.597769 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:36:27.597866 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:36:27.597931 (+ 65us) client_negotiation.cc:769] Sending connection context
0929 10:36:27.597987 (+ 56us) client_negotiation.cc:240] Negotiation successful
0929 10:36:27.598092 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260929 10:36:27.598974 32101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:27.592749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58240 (local address 127.15.82.66:35159)
0929 10:36:27.593591 (+ 842us) server_negotiation.cc:206] Beginning negotiation
0929 10:36:27.593605 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 10:36:27.593629 (+ 24us) server_negotiation.cc:407] Connection header received
0929 10:36:27.593712 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:36:27.593718 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:36:27.593948 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:36:27.594115 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:36:27.594354 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:36:27.595670 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:36:27.596724 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:36:27.597482 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:36:27.598482 (+ 1000us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:36:27.598659 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0929 10:36:27.598714 (+ 55us) server_negotiation.cc:299] Negotiation successful
0929 10:36:27.598811 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":748,"thread_start_us":194,"threads_started":1}
I20260929 10:36:27.599141 32100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:27.592796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.67:44377 (local address 127.0.0.1:37982)
0929 10:36:27.593239 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0929 10:36:27.593252 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 10:36:27.593334 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:36:27.594308 (+ 974us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:36:27.594311 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:36:27.594334 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:36:27.594464 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:36:27.594473 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:36:27.595809 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:36:27.595813 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:36:27.597049 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:36:27.597062 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:36:27.598719 (+ 1657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:36:27.598723 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:36:27.598827 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:36:27.598849 (+ 22us) client_negotiation.cc:769] Sending connection context
0929 10:36:27.598914 (+ 65us) client_negotiation.cc:240] Negotiation successful
0929 10:36:27.598988 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":112,"threads_started":1}
I20260929 10:36:27.599720 31713 tablet_bootstrap.cc:492] T e757f25757a940fba2a32c51bc0ac3bf P 63f8e03d49954851a2b4b48305144ee5: No bootstrap required, opened a new log
I20260929 10:36:27.599740 32102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:27.593232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37982 (local address 127.15.82.67:44377)
0929 10:36:27.593592 (+ 360us) server_negotiation.cc:206] Beginning negotiation
0929 10:36:27.593605 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 10:36:27.593629 (+ 24us) server_negotiation.cc:407] Connection header received
0929 10:36:27.593711 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:36:27.593717 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:36:27.593948 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:36:27.594115 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:36:27.594579 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:36:27.595665 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:36:27.597199 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:36:27.598583 (+ 1384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:36:27.599316 (+ 733us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:36:27.599458 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0929 10:36:27.599521 (+ 63us) server_negotiation.cc:299] Negotiation successful
0929 10:36:27.599608 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":132,"threads_started":1}
I20260929 10:36:27.599831 31713 ts_tablet_manager.cc:1397] T e757f25757a940fba2a32c51bc0ac3bf P 63f8e03d49954851a2b4b48305144ee5: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260929 10:36:27.600219 31713 raft_consensus.cc:348] T e757f25757a940fba2a32c51bc0ac3bf P 63f8e03d49954851a2b4b48305144ee5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6581ebffa3246a987710e539ce43fa3" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 35159 } } peers { permanent_uuid: "04ba93cc78414e468e3bb02139bc71c0" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 44377 } } peers { permanent_uuid: "63f8e03d49954851a2b4b48305144ee5" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 36197 } }
I20260929 10:36:27.600340 31713 raft_consensus.cc:374] T e757f25757a940fba2a32c51bc0ac3bf P 63f8e03d49954851a2b4b48305144ee5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:36:27.600387 31713 raft_consensus.cc:729] T e757f25757a940fba2a32c51bc0ac3bf P 63f8e03d49954851a2b4b48305144ee5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63f8e03d49954851a2b4b48305144ee5, State: Initialized, Role: FOLLOWER
I20260929 10:36:27.600533 31713 consensus_queue.cc:260] T e757f25757a940fba2a32c51bc0ac3bf P 63f8e03d49954851a2b4b48305144ee5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6581ebffa3246a987710e539ce43fa3" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 35159 } } peers { permanent_uuid: "04ba93cc78414e468e3bb02139bc71c0" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 44377 } } peers { permanent_uuid: "63f8e03d49954851a2b4b48305144ee5" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 36197 } }
I20260929 10:36:27.600880 31713 ts_tablet_manager.cc:1428] T e757f25757a940fba2a32c51bc0ac3bf P 63f8e03d49954851a2b4b48305144ee5: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260929 10:36:27.602290 31893 tablet_service.cc:1467] Processing CreateTablet for tablet e757f25757a940fba2a32c51bc0ac3bf (DEFAULT_TABLE table=test-table [id=485a152dedb8432d96e5766c2d7347d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:36:27.602286 32025 tablet_service.cc:1467] Processing CreateTablet for tablet e757f25757a940fba2a32c51bc0ac3bf (DEFAULT_TABLE table=test-table [id=485a152dedb8432d96e5766c2d7347d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:36:27.603442 31893 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e757f25757a940fba2a32c51bc0ac3bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:36:27.603446 32025 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e757f25757a940fba2a32c51bc0ac3bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:36:27.612046 32104 tablet_bootstrap.cc:492] T e757f25757a940fba2a32c51bc0ac3bf P 04ba93cc78414e468e3bb02139bc71c0: Bootstrap starting.
I20260929 10:36:27.612821 32105 tablet_bootstrap.cc:492] T e757f25757a940fba2a32c51bc0ac3bf P e6581ebffa3246a987710e539ce43fa3: Bootstrap starting.
I20260929 10:36:27.615190 32104 tablet_bootstrap.cc:654] T e757f25757a940fba2a32c51bc0ac3bf P 04ba93cc78414e468e3bb02139bc71c0: Neither blocks nor log segments found. Creating new log.
I20260929 10:36:27.615464 32105 tablet_bootstrap.cc:654] T e757f25757a940fba2a32c51bc0ac3bf P e6581ebffa3246a987710e539ce43fa3: Neither blocks nor log segments found. Creating new log.
I20260929 10:36:27.616192 32104 log.cc:824] T e757f25757a940fba2a32c51bc0ac3bf P 04ba93cc78414e468e3bb02139bc71c0: Log is configured to *not* fsync() on all Append() calls
I20260929 10:36:27.616192 32105 log.cc:824] T e757f25757a940fba2a32c51bc0ac3bf P e6581ebffa3246a987710e539ce43fa3: Log is configured to *not* fsync() on all Append() calls
I20260929 10:36:27.618165 32105 tablet_bootstrap.cc:492] T e757f25757a940fba2a32c51bc0ac3bf P e6581ebffa3246a987710e539ce43fa3: No bootstrap required, opened a new log
I20260929 10:36:27.618172 32104 tablet_bootstrap.cc:492] T e757f25757a940fba2a32c51bc0ac3bf P 04ba93cc78414e468e3bb02139bc71c0: No bootstrap required, opened a new log
I20260929 10:36:27.618435 32105 ts_tablet_manager.cc:1397] T e757f25757a940fba2a32c51bc0ac3bf P e6581ebffa3246a987710e539ce43fa3: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260929 10:36:27.618445 32104 ts_tablet_manager.cc:1397] T e757f25757a940fba2a32c51bc0ac3bf P 04ba93cc78414e468e3bb02139bc71c0: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260929 10:36:27.621464 32104 raft_consensus.cc:348] T e757f25757a940fba2a32c51bc0ac3bf P 04ba93cc78414e468e3bb02139bc71c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6581ebffa3246a987710e539ce43fa3" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 35159 } } peers { permanent_uuid: "04ba93cc78414e468e3bb02139bc71c0" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 44377 } } peers { permanent_uuid: "63f8e03d49954851a2b4b48305144ee5" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 36197 } }
I20260929 10:36:27.621461 32105 raft_consensus.cc:348] T e757f25757a940fba2a32c51bc0ac3bf P e6581ebffa3246a987710e539ce43fa3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6581ebffa3246a987710e539ce43fa3" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 35159 } } peers { permanent_uuid: "04ba93cc78414e468e3bb02139bc71c0" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 44377 } } peers { permanent_uuid: "63f8e03d49954851a2b4b48305144ee5" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 36197 } }
I20260929 10:36:27.621778 32104 raft_consensus.cc:374] T e757f25757a940fba2a32c51bc0ac3bf P 04ba93cc78414e468e3bb02139bc71c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:36:27.621779 32105 raft_consensus.cc:374] T e757f25757a940fba2a32c51bc0ac3bf P e6581ebffa3246a987710e539ce43fa3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:36:27.621842 32105 raft_consensus.cc:729] T e757f25757a940fba2a32c51bc0ac3bf P e6581ebffa3246a987710e539ce43fa3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6581ebffa3246a987710e539ce43fa3, State: Initialized, Role: FOLLOWER
I20260929 10:36:27.621841 32104 raft_consensus.cc:729] T e757f25757a940fba2a32c51bc0ac3bf P 04ba93cc78414e468e3bb02139bc71c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04ba93cc78414e468e3bb02139bc71c0, State: Initialized, Role: FOLLOWER
I20260929 10:36:27.622303 32105 consensus_queue.cc:260] T e757f25757a940fba2a32c51bc0ac3bf P e6581ebffa3246a987710e539ce43fa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6581ebffa3246a987710e539ce43fa3" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 35159 } } peers { permanent_uuid: "04ba93cc78414e468e3bb02139bc71c0" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 44377 } } peers { permanent_uuid: "63f8e03d49954851a2b4b48305144ee5" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 36197 } }
I20260929 10:36:27.622316 32104 consensus_queue.cc:260] T e757f25757a940fba2a32c51bc0ac3bf P 04ba93cc78414e468e3bb02139bc71c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6581ebffa3246a987710e539ce43fa3" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 35159 } } peers { permanent_uuid: "04ba93cc78414e468e3bb02139bc71c0" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 44377 } } peers { permanent_uuid: "63f8e03d49954851a2b4b48305144ee5" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 36197 } }
I20260929 10:36:27.623050 31959 heartbeater.cc:503] Master 127.15.82.126:44709 was elected leader, sending a full tablet report...
I20260929 10:36:27.623225 32091 heartbeater.cc:503] Master 127.15.82.126:44709 was elected leader, sending a full tablet report...
I20260929 10:36:27.623346 32104 ts_tablet_manager.cc:1428] T e757f25757a940fba2a32c51bc0ac3bf P 04ba93cc78414e468e3bb02139bc71c0: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260929 10:36:27.623346 32105 ts_tablet_manager.cc:1428] T e757f25757a940fba2a32c51bc0ac3bf P e6581ebffa3246a987710e539ce43fa3: Time spent starting tablet: real 0.005s user 0.001s sys 0.003s
W20260929 10:36:27.635234 31822 tablet.cc:2369] T e757f25757a940fba2a32c51bc0ac3bf P 63f8e03d49954851a2b4b48305144ee5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 10:36:27.701109 32109 raft_consensus.cc:482] T e757f25757a940fba2a32c51bc0ac3bf P 04ba93cc78414e468e3bb02139bc71c0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:36:27.701340 32109 raft_consensus.cc:504] T e757f25757a940fba2a32c51bc0ac3bf P 04ba93cc78414e468e3bb02139bc71c0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6581ebffa3246a987710e539ce43fa3" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 35159 } } peers { permanent_uuid: "04ba93cc78414e468e3bb02139bc71c0" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 44377 } } peers { permanent_uuid: "63f8e03d49954851a2b4b48305144ee5" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 36197 } }
I20260929 10:36:27.702673 32109 leader_election.cc:288] T e757f25757a940fba2a32c51bc0ac3bf P 04ba93cc78414e468e3bb02139bc71c0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e6581ebffa3246a987710e539ce43fa3 (127.15.82.66:35159), 63f8e03d49954851a2b4b48305144ee5 (127.15.82.65:36197)
I20260929 10:36:27.706851 32093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:27.702897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.66:35159 (local address 127.15.82.67:60041)
0929 10:36:27.703025 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0929 10:36:27.703041 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 10:36:27.703132 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:36:27.703523 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:36:27.703525 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:36:27.703545 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:36:27.703660 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:36:27.703665 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:36:27.704634 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:36:27.704637 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:36:27.705541 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:36:27.705550 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:36:27.706360 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:36:27.706364 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:36:27.706509 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:36:27.706527 (+ 18us) client_negotiation.cc:769] Sending connection context
0929 10:36:27.706629 (+ 102us) client_negotiation.cc:240] Negotiation successful
0929 10:36:27.706701 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260929 10:36:27.706851 32101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:27.702977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.67:60041 (local address 127.15.82.66:35159)
0929 10:36:27.703100 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0929 10:36:27.703104 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:36:27.703119 (+ 15us) server_negotiation.cc:407] Connection header received
0929 10:36:27.703263 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:36:27.703267 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:36:27.703367 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:36:27.703499 (+ 132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:36:27.703793 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:36:27.704498 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:36:27.705659 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:36:27.706236 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:36:27.706373 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:36:27.706453 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0929 10:36:27.706644 (+ 191us) server_negotiation.cc:299] Negotiation successful
0929 10:36:27.706728 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260929 10:36:27.707489 31913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e757f25757a940fba2a32c51bc0ac3bf" candidate_uuid: "04ba93cc78414e468e3bb02139bc71c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6581ebffa3246a987710e539ce43fa3" is_pre_election: true
I20260929 10:36:27.707595 32110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:27.703019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.67:56609 (local address 127.15.82.65:36197)
0929 10:36:27.703274 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0929 10:36:27.703278 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:36:27.703830 (+ 552us) server_negotiation.cc:407] Connection header received
0929 10:36:27.703915 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:36:27.703917 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:36:27.703972 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:36:27.704108 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:36:27.704378 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:36:27.704930 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:36:27.706127 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:36:27.706732 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:36:27.706887 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:36:27.707058 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0929 10:36:27.707399 (+ 341us) server_negotiation.cc:299] Negotiation successful
0929 10:36:27.707476 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":105,"threads_started":1}
I20260929 10:36:27.707603 32111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:27.702910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.65:36197 (local address 127.15.82.67:56609)
0929 10:36:27.703709 (+ 799us) negotiation.cc:107] Waiting for socket to connect
0929 10:36:27.703720 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:36:27.703835 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:36:27.704138 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:36:27.704141 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:36:27.704159 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:36:27.704264 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:36:27.704269 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:36:27.705041 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:36:27.705045 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:36:27.706005 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:36:27.706014 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:36:27.706877 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:36:27.706881 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:36:27.707253 (+ 372us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:36:27.707273 (+ 20us) client_negotiation.cc:769] Sending connection context
0929 10:36:27.707387 (+ 114us) client_negotiation.cc:240] Negotiation successful
0929 10:36:27.707459 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":745,"mutex_wait_us":332,"thread_start_us":195,"threads_started":1}
I20260929 10:36:27.707835 31913 raft_consensus.cc:2454] T e757f25757a940fba2a32c51bc0ac3bf P e6581ebffa3246a987710e539ce43fa3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04ba93cc78414e468e3bb02139bc71c0 in term 0.
I20260929 10:36:27.708359 31981 leader_election.cc:302] T e757f25757a940fba2a32c51bc0ac3bf P 04ba93cc78414e468e3bb02139bc71c0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04ba93cc78414e468e3bb02139bc71c0, e6581ebffa3246a987710e539ce43fa3; no voters:
I20260929 10:36:27.708664 32109 raft_consensus.cc:2781] T e757f25757a940fba2a32c51bc0ac3bf P 04ba93cc78414e468e3bb02139bc71c0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:36:27.708659 31772 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e757f25757a940fba2a32c51bc0ac3bf" candidate_uuid: "04ba93cc78414e468e3bb02139bc71c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63f8e03d49954851a2b4b48305144ee5" is_pre_election: true
I20260929 10:36:27.708762 32109 raft_consensus.cc:482] T e757f25757a940fba2a32c51bc0ac3bf P 04ba93cc78414e468e3bb02139bc71c0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:36:27.708810 32109 raft_consensus.cc:3037] T e757f25757a940fba2a32c51bc0ac3bf P 04ba93cc78414e468e3bb02139bc71c0 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:36:27.708956 31772 raft_consensus.cc:2454] T e757f25757a940fba2a32c51bc0ac3bf P 63f8e03d49954851a2b4b48305144ee5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04ba93cc78414e468e3bb02139bc71c0 in term 0.
I20260929 10:36:27.710011 32109 raft_consensus.cc:504] T e757f25757a940fba2a32c51bc0ac3bf P 04ba93cc78414e468e3bb02139bc71c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6581ebffa3246a987710e539ce43fa3" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 35159 } } peers { permanent_uuid: "04ba93cc78414e468e3bb02139bc71c0" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 44377 } } peers { permanent_uuid: "63f8e03d49954851a2b4b48305144ee5" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 36197 } }
I20260929 10:36:27.710325 32109 leader_election.cc:288] T e757f25757a940fba2a32c51bc0ac3bf P 04ba93cc78414e468e3bb02139bc71c0 [CANDIDATE]: Term 1 election: Requested vote from peers e6581ebffa3246a987710e539ce43fa3 (127.15.82.66:35159), 63f8e03d49954851a2b4b48305144ee5 (127.15.82.65:36197)
I20260929 10:36:27.710678 31772 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e757f25757a940fba2a32c51bc0ac3bf" candidate_uuid: "04ba93cc78414e468e3bb02139bc71c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63f8e03d49954851a2b4b48305144ee5"
I20260929 10:36:27.710678 31913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e757f25757a940fba2a32c51bc0ac3bf" candidate_uuid: "04ba93cc78414e468e3bb02139bc71c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6581ebffa3246a987710e539ce43fa3"
I20260929 10:36:27.710805 31772 raft_consensus.cc:3037] T e757f25757a940fba2a32c51bc0ac3bf P 63f8e03d49954851a2b4b48305144ee5 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:36:27.710811 31913 raft_consensus.cc:3037] T e757f25757a940fba2a32c51bc0ac3bf P e6581ebffa3246a987710e539ce43fa3 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:36:27.712033 31913 raft_consensus.cc:2454] T e757f25757a940fba2a32c51bc0ac3bf P e6581ebffa3246a987710e539ce43fa3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04ba93cc78414e468e3bb02139bc71c0 in term 1.
I20260929 10:36:27.712033 31772 raft_consensus.cc:2454] T e757f25757a940fba2a32c51bc0ac3bf P 63f8e03d49954851a2b4b48305144ee5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04ba93cc78414e468e3bb02139bc71c0 in term 1.
I20260929 10:36:27.712313 31978 leader_election.cc:302] T e757f25757a940fba2a32c51bc0ac3bf P 04ba93cc78414e468e3bb02139bc71c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 04ba93cc78414e468e3bb02139bc71c0, 63f8e03d49954851a2b4b48305144ee5, e6581ebffa3246a987710e539ce43fa3; no voters:
I20260929 10:36:27.712514 32109 raft_consensus.cc:2781] T e757f25757a940fba2a32c51bc0ac3bf P 04ba93cc78414e468e3bb02139bc71c0 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:36:27.712680 32109 raft_consensus.cc:686] T e757f25757a940fba2a32c51bc0ac3bf P 04ba93cc78414e468e3bb02139bc71c0 [term 1 LEADER]: Becoming Leader. State: Replica: 04ba93cc78414e468e3bb02139bc71c0, State: Running, Role: LEADER
I20260929 10:36:27.713047 32109 consensus_queue.cc:237] T e757f25757a940fba2a32c51bc0ac3bf P 04ba93cc78414e468e3bb02139bc71c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6581ebffa3246a987710e539ce43fa3" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 35159 } } peers { permanent_uuid: "04ba93cc78414e468e3bb02139bc71c0" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 44377 } } peers { permanent_uuid: "63f8e03d49954851a2b4b48305144ee5" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 36197 } }
I20260929 10:36:27.716310 31633 catalog_manager.cc:5495] T e757f25757a940fba2a32c51bc0ac3bf P 04ba93cc78414e468e3bb02139bc71c0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 04ba93cc78414e468e3bb02139bc71c0 (127.15.82.67). New cstate: current_term: 1 leader_uuid: "04ba93cc78414e468e3bb02139bc71c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6581ebffa3246a987710e539ce43fa3" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 35159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "04ba93cc78414e468e3bb02139bc71c0" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 44377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "63f8e03d49954851a2b4b48305144ee5" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 36197 } health_report { overall_health: UNKNOWN } } }
I20260929 10:36:27.727149 32102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:27.723654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37996 (local address 127.15.82.67:44377)
0929 10:36:27.723783 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0929 10:36:27.723789 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 10:36:27.723804 (+ 15us) server_negotiation.cc:407] Connection header received
0929 10:36:27.723991 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:36:27.723994 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:36:27.724045 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:36:27.724151 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:36:27.724379 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:36:27.724867 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:36:27.725705 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:36:27.725923 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:36:27.726024 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:36:27.726326 (+ 302us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:36:27.726784 (+ 458us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:36:27.726923 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0929 10:36:27.726959 (+ 36us) server_negotiation.cc:299] Negotiation successful
0929 10:36:27.727011 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260929 10:36:27.732947 31913 raft_consensus.cc:1261] T e757f25757a940fba2a32c51bc0ac3bf P e6581ebffa3246a987710e539ce43fa3 [term 1 FOLLOWER]: Refusing update from remote peer 04ba93cc78414e468e3bb02139bc71c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:36:27.732944 31772 raft_consensus.cc:1261] T e757f25757a940fba2a32c51bc0ac3bf P 63f8e03d49954851a2b4b48305144ee5 [term 1 FOLLOWER]: Refusing update from remote peer 04ba93cc78414e468e3bb02139bc71c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:36:27.733642 32112 consensus_queue.cc:1035] T e757f25757a940fba2a32c51bc0ac3bf P 04ba93cc78414e468e3bb02139bc71c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "63f8e03d49954851a2b4b48305144ee5" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 36197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:36:27.733839 32109 consensus_queue.cc:1035] T e757f25757a940fba2a32c51bc0ac3bf P 04ba93cc78414e468e3bb02139bc71c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6581ebffa3246a987710e539ce43fa3" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 35159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:36:27.751401 31631 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55186:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:36:27.751564 31631 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55186:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:36:27.753911 31631 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8fb3386c28a34308a393436c0d1cde01: Sending DeleteTablet for 3 replicas of tablet e757f25757a940fba2a32c51bc0ac3bf
I20260929 10:36:27.754668 31893 tablet_service.cc:1514] Processing DeleteTablet for tablet e757f25757a940fba2a32c51bc0ac3bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:36:27 UTC) from {username='kudu', principal='kudu/127.15.82.126@KRBTEST.COM'} at 127.0.0.1:58240
I20260929 10:36:27.754787 31755 tablet_service.cc:1514] Processing DeleteTablet for tablet e757f25757a940fba2a32c51bc0ac3bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:36:27 UTC) from {username='kudu', principal='kudu/127.15.82.126@KRBTEST.COM'} at 127.0.0.1:34220
I20260929 10:36:27.754838 32025 tablet_service.cc:1514] Processing DeleteTablet for tablet e757f25757a940fba2a32c51bc0ac3bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:36:27 UTC) from {username='kudu', principal='kudu/127.15.82.126@KRBTEST.COM'} at 127.0.0.1:37982
I20260929 10:36:27.755227 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 31671
I20260929 10:36:27.755283 32124 tablet_replica.cc:330] T e757f25757a940fba2a32c51bc0ac3bf P 63f8e03d49954851a2b4b48305144ee5: stopping tablet replica
I20260929 10:36:27.755498 32125 tablet_replica.cc:330] T e757f25757a940fba2a32c51bc0ac3bf P e6581ebffa3246a987710e539ce43fa3: stopping tablet replica
I20260929 10:36:27.755697 32126 tablet_replica.cc:330] T e757f25757a940fba2a32c51bc0ac3bf P 04ba93cc78414e468e3bb02139bc71c0: stopping tablet replica
I20260929 10:36:27.756057 32126 raft_consensus.cc:2229] T e757f25757a940fba2a32c51bc0ac3bf P 04ba93cc78414e468e3bb02139bc71c0 [term 1 LEADER]: Raft consensus shutting down.
I20260929 10:36:27.756415 32125 raft_consensus.cc:2229] T e757f25757a940fba2a32c51bc0ac3bf P e6581ebffa3246a987710e539ce43fa3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:36:27.756799 32125 raft_consensus.cc:2258] T e757f25757a940fba2a32c51bc0ac3bf P e6581ebffa3246a987710e539ce43fa3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:36:27.757244 32126 raft_consensus.cc:2258] T e757f25757a940fba2a32c51bc0ac3bf P 04ba93cc78414e468e3bb02139bc71c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:36:27.759140 32126 ts_tablet_manager.cc:1905] T e757f25757a940fba2a32c51bc0ac3bf P 04ba93cc78414e468e3bb02139bc71c0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:36:27.759155 32125 ts_tablet_manager.cc:1905] T e757f25757a940fba2a32c51bc0ac3bf P e6581ebffa3246a987710e539ce43fa3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:36:27.761893 32125 ts_tablet_manager.cc:1918] T e757f25757a940fba2a32c51bc0ac3bf P e6581ebffa3246a987710e539ce43fa3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:36:27.762003 32125 log.cc:1196] T e757f25757a940fba2a32c51bc0ac3bf P e6581ebffa3246a987710e539ce43fa3: Deleting WAL directory at /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-1/wal/wals/e757f25757a940fba2a32c51bc0ac3bf
I20260929 10:36:27.762354 32125 ts_tablet_manager.cc:1939] T e757f25757a940fba2a32c51bc0ac3bf P e6581ebffa3246a987710e539ce43fa3: Deleting consensus metadata
I20260929 10:36:27.762830 32126 ts_tablet_manager.cc:1918] T e757f25757a940fba2a32c51bc0ac3bf P 04ba93cc78414e468e3bb02139bc71c0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:36:27.763003 32126 log.cc:1196] T e757f25757a940fba2a32c51bc0ac3bf P 04ba93cc78414e468e3bb02139bc71c0: Deleting WAL directory at /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-2/wal/wals/e757f25757a940fba2a32c51bc0ac3bf
W20260929 10:36:27.763128 31617 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.82.65:36197 (error 108)
I20260929 10:36:27.763460 32126 ts_tablet_manager.cc:1939] T e757f25757a940fba2a32c51bc0ac3bf P 04ba93cc78414e468e3bb02139bc71c0: Deleting consensus metadata
I20260929 10:36:27.763613 31620 catalog_manager.cc:4841] TS e6581ebffa3246a987710e539ce43fa3 (127.15.82.66:35159): tablet e757f25757a940fba2a32c51bc0ac3bf (table test-table [id=485a152dedb8432d96e5766c2d7347d4]) successfully deleted
I20260929 10:36:27.764335 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 31825
I20260929 10:36:27.764693 32100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:27.764391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.65:36197 (local address 127.0.0.1:34222)
0929 10:36:27.764525 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0929 10:36:27.764598 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.65:36197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260929 10:36:27.765612 31620 catalog_manager.cc:4841] TS 04ba93cc78414e468e3bb02139bc71c0 (127.15.82.67:44377): tablet e757f25757a940fba2a32c51bc0ac3bf (table test-table [id=485a152dedb8432d96e5766c2d7347d4]) successfully deleted
W20260929 10:36:27.765591 31617 catalog_manager.cc:4584] TS 63f8e03d49954851a2b4b48305144ee5 (127.15.82.65:36197): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e757f25757a940fba2a32c51bc0ac3bf: Network error: Client connection negotiation failed: client connection to 127.15.82.65:36197: connect: Connection refused (error 111)
I20260929 10:36:27.770370 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 31962
I20260929 10:36:27.776154 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 31600
I20260929 10:36:47.782910 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:44709
--webserver_interface=127.15.82.126
--webserver_port=36519
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab
--principal=kudu/127.15.82.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:34999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.82.126:44709
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:36:47.891681 32128 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:36:47.891934 32128 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:36:47.891974 32128 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:36:47.895967 32128 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260929 10:36:47.896013 32128 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260929 10:36:47.896030 32128 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260929 10:36:47.896044 32128 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:36:47.896062 32128 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:36:47.896127 32128 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:36:47.896157 32128 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:36:47.896186 32128 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:36:47.901360 32128 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:34999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.15.82.126:44709
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab
--principal=kudu/127.15.82.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:44709
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.82.126
--webserver_port=36519
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32128
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:36:47.902427 32128 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:36:47.903451 32128 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 10:36:47.912141 32128 server_base.cc:1034] running on GCE node
W20260929 10:36:47.912145 32136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:36:47.912197 32134 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:36:47.912228 32133 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:36:47.913110 32128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:36:47.914136 32128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:36:47.915314 32128 hybrid_clock.cc:648] HybridClock initialized: now 1790678207915279 us; error 47 us; skew 500 ppm
Sep 29 10:36:47 dist-test-slave-vg89 krb5kdc[31052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678207, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:36:47.918111 32128 init.cc:377] Logged in from keytab as kudu/127.15.82.126@KRBTEST.COM (short username kudu)
I20260929 10:36:47.919056 32128 webserver.cc:460] Webserver started at http://127.15.82.126:36519/ using document root <none> and password file <none>
I20260929 10:36:47.919610 32128 fs_manager.cc:359] Metadata directory not provided
I20260929 10:36:47.919703 32128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:36:47.923471 32128 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260929 10:36:47.925513 32143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:36:47.926821 32128 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 10:36:47.926961 32128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/master-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/master-0/wal
uuid: "8fb3386c28a34308a393436c0d1cde01"
format_stamp: "Formatted at 2026-09-29 10:36:05 on dist-test-slave-vg89"
server_key: "1542023ca7ac7ab95c4e1ede5b9e973f"
server_key_iv: "939e30e658d7c6464f336ce80cee90ae"
server_key_version: "encryptionkey@0"
I20260929 10:36:47.927485 32128 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:36:47.959496 32128 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:36:47.962955 32128 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:36:47.963165 32128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:36:47.970172 32128 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.126:44709
I20260929 10:36:47.970486 32195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.126:44709 every 8 connection(s)
I20260929 10:36:47.971323 32128 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
I20260929 10:36:47.975514 32196 sys_catalog.cc:263] Verifying existing consensus state
I20260929 10:36:47.977247 32196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fb3386c28a34308a393436c0d1cde01: Bootstrap starting.
I20260929 10:36:47.980361 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 32128
I20260929 10:36:47.980887 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.65:36197
--local_ip_for_outbound_sockets=127.15.82.65
--tserver_master_addrs=127.15.82.126:44709
--webserver_port=36785
--webserver_interface=127.15.82.65
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab
--principal=kudu/127.15.82.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:34999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 10:36:47.995397 32196 log.cc:824] T 00000000000000000000000000000000 P 8fb3386c28a34308a393436c0d1cde01: Log is configured to *not* fsync() on all Append() calls
I20260929 10:36:48.006590 32196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fb3386c28a34308a393436c0d1cde01: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 10:36:48.006989 32196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fb3386c28a34308a393436c0d1cde01: Bootstrap complete.
I20260929 10:36:48.011121 32196 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8fb3386c28a34308a393436c0d1cde01 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fb3386c28a34308a393436c0d1cde01" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 44709 } }
I20260929 10:36:48.011494 32196 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8fb3386c28a34308a393436c0d1cde01 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fb3386c28a34308a393436c0d1cde01, State: Initialized, Role: FOLLOWER
I20260929 10:36:48.011977 32196 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8fb3386c28a34308a393436c0d1cde01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fb3386c28a34308a393436c0d1cde01" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 44709 } }
I20260929 10:36:48.012094 32196 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8fb3386c28a34308a393436c0d1cde01 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:36:48.012181 32196 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8fb3386c28a34308a393436c0d1cde01 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:36:48.012295 32196 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8fb3386c28a34308a393436c0d1cde01 [term 2 FOLLOWER]: Advancing to term 3
I20260929 10:36:48.014204 32196 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8fb3386c28a34308a393436c0d1cde01 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fb3386c28a34308a393436c0d1cde01" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 44709 } }
I20260929 10:36:48.014559 32196 leader_election.cc:302] T 00000000000000000000000000000000 P 8fb3386c28a34308a393436c0d1cde01 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8fb3386c28a34308a393436c0d1cde01; no voters:
I20260929 10:36:48.014909 32196 leader_election.cc:288] T 00000000000000000000000000000000 P 8fb3386c28a34308a393436c0d1cde01 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260929 10:36:48.015013 32200 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8fb3386c28a34308a393436c0d1cde01 [term 3 FOLLOWER]: Leader election won for term 3
I20260929 10:36:48.015213 32200 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8fb3386c28a34308a393436c0d1cde01 [term 3 LEADER]: Becoming Leader. State: Replica: 8fb3386c28a34308a393436c0d1cde01, State: Running, Role: LEADER
I20260929 10:36:48.015589 32200 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8fb3386c28a34308a393436c0d1cde01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fb3386c28a34308a393436c0d1cde01" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 44709 } }
I20260929 10:36:48.016124 32196 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8fb3386c28a34308a393436c0d1cde01 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:36:48.016498 32200 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fb3386c28a34308a393436c0d1cde01 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8fb3386c28a34308a393436c0d1cde01. Latest consensus state: current_term: 3 leader_uuid: "8fb3386c28a34308a393436c0d1cde01" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fb3386c28a34308a393436c0d1cde01" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 44709 } } }
I20260929 10:36:48.016611 32200 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fb3386c28a34308a393436c0d1cde01 [sys.catalog]: This master's current role is: LEADER
I20260929 10:36:48.017005 32201 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fb3386c28a34308a393436c0d1cde01 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "8fb3386c28a34308a393436c0d1cde01" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fb3386c28a34308a393436c0d1cde01" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 44709 } } }
I20260929 10:36:48.017176 32201 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fb3386c28a34308a393436c0d1cde01 [sys.catalog]: This master's current role is: LEADER
I20260929 10:36:48.021009 32215 catalog_manager.cc:1259] Loaded cluster ID: b3abbf64430d41ac8eec98adc906db15
I20260929 10:36:48.021090 32215 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8fb3386c28a34308a393436c0d1cde01: loading cluster ID for follower catalog manager: success
I20260929 10:36:48.022936 32215 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8fb3386c28a34308a393436c0d1cde01: acquiring CA information for follower catalog manager: success
F20260929 10:36:48.023844 32215 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f5ca83409ad google::LogMessage::Fail() at ??:0
@ 0x7f5ca834388a google::LogMessage::SendToLog() at ??:0
@ 0x7f5ca83404de google::LogMessage::Flush() at ??:0
@ 0x7f5ca8341a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f5cab00418d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f5cb2ca5989 kudu::master::CatalogManager::LoadTspkEntries() at ??:0
@ 0x7f5cb2c82023 kudu::master::CatalogManager::PrepareFollowerTokenVerifier() at ??:0
@ 0x7f5cb2c8254b kudu::master::CatalogManager::PrepareFollower() at ??:0
@ 0x7f5cb2c7b652 kudu::master::CatalogManagerBgTasks::Run() at ??:0
@ 0x7f5cb2c7a9e9 _ZZN4kudu6master21CatalogManagerBgTasks4InitEvENKUlvE_clEv at ??:0
@ 0x7f5cb2cc1541 _ZNSt17_Function_handlerIFvvEZN4kudu6master21CatalogManagerBgTasks4InitEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f5caa583512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f5cabbc76db start_thread at ??:0
@ 0x7f5ca6e3e61f clone at ??:0
W20260929 10:36:48.112986 32198 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:36:48.113283 32198 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:36:48.113353 32198 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:36:48.117558 32198 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:36:48.117640 32198 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:36:48.117729 32198 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.65
I20260929 10:36:48.123174 32198 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:34999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab
--principal=kudu/127.15.82.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.65:36197
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.82.65
--webserver_port=36785
--tserver_master_addrs=127.15.82.126:44709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32198
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.65
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:36:48.124279 32198 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:36:48.125353 32198 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:36:48.135434 32236 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:36:48.135437 32237 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:36:48.135746 32198 server_base.cc:1034] running on GCE node
W20260929 10:36:48.135566 32239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:36:48.136322 32198 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:36:48.137167 32198 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:36:48.138370 32198 hybrid_clock.cc:648] HybridClock initialized: now 1790678208138329 us; error 48 us; skew 500 ppm
Sep 29 10:36:48 dist-test-slave-vg89 krb5kdc[31052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678208, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:36:48.141853 32198 init.cc:377] Logged in from keytab as kudu/127.15.82.65@KRBTEST.COM (short username kudu)
I20260929 10:36:48.143081 32198 webserver.cc:460] Webserver started at http://127.15.82.65:36785/ using document root <none> and password file <none>
I20260929 10:36:48.143814 32198 fs_manager.cc:359] Metadata directory not provided
I20260929 10:36:48.143927 32198 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:36:48.149057 32198 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260929 10:36:48.151713 32246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:36:48.153261 32198 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 10:36:48.153384 32198 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-0/wal
uuid: "63f8e03d49954851a2b4b48305144ee5"
format_stamp: "Formatted at 2026-09-29 10:36:05 on dist-test-slave-vg89"
server_key: "3572e645b2fd051b89fa5a6e2866f4ac"
server_key_iv: "3a2d499a1385e60427468602d7a17cdc"
server_key_version: "encryptionkey@0"
I20260929 10:36:48.154072 32198 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:36:48.181248 32198 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:36:48.185231 32198 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:36:48.185436 32198 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:36:48.186200 32198 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:36:48.187608 32253 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260929 10:36:48.191992 32198 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260929 10:36:48.192073 32198 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260929 10:36:48.192144 32198 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260929 10:36:48.194514 32198 ts_tablet_manager.cc:610] Registered 1 tablets
I20260929 10:36:48.194576 32198 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.002s sys 0.000s
I20260929 10:36:48.194953 32253 tablet_bootstrap.cc:492] T e757f25757a940fba2a32c51bc0ac3bf P 63f8e03d49954851a2b4b48305144ee5: Bootstrap starting.
I20260929 10:36:48.205565 32198 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.65:36197
I20260929 10:36:48.205822 32360 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.65:36197 every 8 connection(s)
I20260929 10:36:48.206895 32198 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
I20260929 10:36:48.208390 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 32198
I20260929 10:36:48.208913 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.66:35159
--local_ip_for_outbound_sockets=127.15.82.66
--tserver_master_addrs=127.15.82.126:44709
--webserver_port=33533
--webserver_interface=127.15.82.66
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab
--principal=kudu/127.15.82.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:34999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 10:36:48.219566 32253 log.cc:824] T e757f25757a940fba2a32c51bc0ac3bf P 63f8e03d49954851a2b4b48305144ee5: Log is configured to *not* fsync() on all Append() calls
I20260929 10:36:48.228868 32253 tablet_bootstrap.cc:492] T e757f25757a940fba2a32c51bc0ac3bf P 63f8e03d49954851a2b4b48305144ee5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 10:36:48.229324 32253 tablet_bootstrap.cc:492] T e757f25757a940fba2a32c51bc0ac3bf P 63f8e03d49954851a2b4b48305144ee5: Bootstrap complete.
I20260929 10:36:48.230813 32253 ts_tablet_manager.cc:1397] T e757f25757a940fba2a32c51bc0ac3bf P 63f8e03d49954851a2b4b48305144ee5: Time spent bootstrapping tablet: real 0.036s user 0.011s sys 0.020s
I20260929 10:36:48.234093 32253 raft_consensus.cc:348] T e757f25757a940fba2a32c51bc0ac3bf P 63f8e03d49954851a2b4b48305144ee5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6581ebffa3246a987710e539ce43fa3" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 35159 } } peers { permanent_uuid: "04ba93cc78414e468e3bb02139bc71c0" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 44377 } } peers { permanent_uuid: "63f8e03d49954851a2b4b48305144ee5" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 36197 } }
I20260929 10:36:48.234673 32253 raft_consensus.cc:729] T e757f25757a940fba2a32c51bc0ac3bf P 63f8e03d49954851a2b4b48305144ee5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63f8e03d49954851a2b4b48305144ee5, State: Initialized, Role: FOLLOWER
I20260929 10:36:48.235209 32253 consensus_queue.cc:260] T e757f25757a940fba2a32c51bc0ac3bf P 63f8e03d49954851a2b4b48305144ee5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6581ebffa3246a987710e539ce43fa3" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 35159 } } peers { permanent_uuid: "04ba93cc78414e468e3bb02139bc71c0" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 44377 } } peers { permanent_uuid: "63f8e03d49954851a2b4b48305144ee5" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 36197 } }
I20260929 10:36:48.236325 32253 ts_tablet_manager.cc:1428] T e757f25757a940fba2a32c51bc0ac3bf P 63f8e03d49954851a2b4b48305144ee5: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
W20260929 10:36:48.251305 32363 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:36:48.209343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:44709 (local address 127.15.82.65:59707)
0929 10:36:48.210971 (+ 1628us) negotiation.cc:107] Waiting for socket to connect
0929 10:36:48.211033 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0929 10:36:48.212110 (+ 1077us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:36:48.250305 (+ 38195us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:44709: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":1370,"thread_start_us":146,"threads_started":1}
W20260929 10:36:48.251914 32361 heartbeater.cc:644] Failed to heartbeat to 127.15.82.126:44709 (0 consecutive failures): Network error: Failed to ping master at 127.15.82.126:44709: Client connection negotiation failed: client connection to 127.15.82.126:44709: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260929 10:36:48.252635 32363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:48.252377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:44709 (local address 127.15.82.65:39809)
0929 10:36:48.252506 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0929 10:36:48.252581 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260929 10:36:48.253351 32363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:48.253147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:44709 (local address 127.15.82.65:41621)
0929 10:36:48.253252 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0929 10:36:48.253305 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260929 10:36:48.253551 32361 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 10:36:48.334667 32364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:36:48.334980 32364 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:36:48.335076 32364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:36:48.339308 32364 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:36:48.339401 32364 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:36:48.339493 32364 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.66
I20260929 10:36:48.345116 32364 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:34999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab
--principal=kudu/127.15.82.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.66:35159
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.82.66
--webserver_port=33533
--tserver_master_addrs=127.15.82.126:44709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32364
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.66
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:36:48.346309 32364 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:36:48.347446 32364 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:36:48.357497 32371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:36:48.357470 32372 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:36:48.357672 32364 server_base.cc:1034] running on GCE node
W20260929 10:36:48.357569 32374 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:36:48.358309 32364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:36:48.359216 32364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:36:48.360442 32364 hybrid_clock.cc:648] HybridClock initialized: now 1790678208360404 us; error 50 us; skew 500 ppm
Sep 29 10:36:48 dist-test-slave-vg89 krb5kdc[31052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678208, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:36:48.363487 32364 init.cc:377] Logged in from keytab as kudu/127.15.82.66@KRBTEST.COM (short username kudu)
I20260929 10:36:48.364401 32364 webserver.cc:460] Webserver started at http://127.15.82.66:33533/ using document root <none> and password file <none>
I20260929 10:36:48.364992 32364 fs_manager.cc:359] Metadata directory not provided
I20260929 10:36:48.365085 32364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:36:48.369196 32364 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260929 10:36:48.371534 32381 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:36:48.372924 32364 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 10:36:48.373085 32364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-1/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-1/wal
uuid: "e6581ebffa3246a987710e539ce43fa3"
format_stamp: "Formatted at 2026-09-29 10:36:06 on dist-test-slave-vg89"
server_key: "0bfe256a079924566e473b8a3bd241f8"
server_key_iv: "8d5aeb5922cc2793b1b2e4164c640eff"
server_key_version: "encryptionkey@0"
I20260929 10:36:48.373682 32364 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:36:48.391996 32364 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:36:48.395210 32364 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:36:48.395437 32364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:36:48.396059 32364 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:36:48.396926 32364 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:36:48.396998 32364 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:36:48.397073 32364 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:36:48.397117 32364 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:36:48.406625 32364 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.66:35159
I20260929 10:36:48.406685 32494 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.66:35159 every 8 connection(s)
I20260929 10:36:48.407702 32364 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
I20260929 10:36:48.409266 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 32364
I20260929 10:36:48.409744 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.67:44377
--local_ip_for_outbound_sockets=127.15.82.67
--tserver_master_addrs=127.15.82.126:44709
--webserver_port=36695
--webserver_interface=127.15.82.67
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab
--principal=kudu/127.15.82.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:34999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 10:36:48.412104 32497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:48.410259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:44709 (local address 127.15.82.66:47337)
0929 10:36:48.410800 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0929 10:36:48.411364 (+ 564us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":182,"threads_started":1}
W20260929 10:36:48.412590 32495 heartbeater.cc:644] Failed to heartbeat to 127.15.82.126:44709 (0 consecutive failures): Network error: Failed to ping master at 127.15.82.126:44709: Client connection negotiation failed: client connection to 127.15.82.126:44709: connect: Connection refused (error 111)
I20260929 10:36:48.413240 32497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:48.412977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:44709 (local address 127.15.82.66:47681)
0929 10:36:48.413096 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0929 10:36:48.413175 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260929 10:36:48.414150 32497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:48.413673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:44709 (local address 127.15.82.66:40245)
0929 10:36:48.413982 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0929 10:36:48.414053 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213}
W20260929 10:36:48.414367 32495 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 10:36:48.531430 32498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:36:48.531741 32498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:36:48.531787 32498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:36:48.536019 32498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:36:48.536103 32498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:36:48.536171 32498 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.67
I20260929 10:36:48.541767 32498 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:34999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab
--principal=kudu/127.15.82.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.67:44377
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.82.67
--webserver_port=36695
--tserver_master_addrs=127.15.82.126:44709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32498
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.67
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:36:48.543346 32498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:36:48.544708 32498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:36:48.555579 32503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:36:48.555579 32504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:36:48.555619 32506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:36:48.556166 32498 server_base.cc:1034] running on GCE node
I20260929 10:36:48.556654 32498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:36:48.557386 32498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:36:48.558594 32498 hybrid_clock.cc:648] HybridClock initialized: now 1790678208558554 us; error 57 us; skew 500 ppm
Sep 29 10:36:48 dist-test-slave-vg89 krb5kdc[31052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678208, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:36:48.561879 32498 init.cc:377] Logged in from keytab as kudu/127.15.82.67@KRBTEST.COM (short username kudu)
I20260929 10:36:48.563177 32498 webserver.cc:460] Webserver started at http://127.15.82.67:36695/ using document root <none> and password file <none>
I20260929 10:36:48.564020 32498 fs_manager.cc:359] Metadata directory not provided
I20260929 10:36:48.564116 32498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:36:48.569931 32498 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260929 10:36:48.572515 32513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:36:48.574137 32498 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 10:36:48.574270 32498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-2/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-2/wal
uuid: "04ba93cc78414e468e3bb02139bc71c0"
format_stamp: "Formatted at 2026-09-29 10:36:06 on dist-test-slave-vg89"
server_key: "91fdf326a1efd5e729607d530e8a9406"
server_key_iv: "50bd17be0b3dda8ae4f58dabbeaebb19"
server_key_version: "encryptionkey@0"
I20260929 10:36:48.575016 32498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:36:48.591329 32498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:36:48.594785 32498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:36:48.595014 32498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:36:48.595809 32498 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:36:48.597114 32498 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:36:48.597169 32498 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:36:48.597247 32498 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:36:48.597285 32498 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:36:48.607760 32498 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.67:44377
I20260929 10:36:48.607784 32626 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.67:44377 every 8 connection(s)
I20260929 10:36:48.608964 32498 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
I20260929 10:36:48.613045 32629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:48.611346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:44709 (local address 127.15.82.67:59523)
0929 10:36:48.612077 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0929 10:36:48.612225 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":121,"threads_started":1}
W20260929 10:36:48.613554 32627 heartbeater.cc:644] Failed to heartbeat to 127.15.82.126:44709 (0 consecutive failures): Network error: Failed to ping master at 127.15.82.126:44709: Client connection negotiation failed: client connection to 127.15.82.126:44709: connect: Connection refused (error 111)
I20260929 10:36:48.614219 32629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:48.613966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:44709 (local address 127.15.82.67:58601)
0929 10:36:48.614096 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0929 10:36:48.614161 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260929 10:36:48.614967 32629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:48.614694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:44709 (local address 127.15.82.67:51803)
0929 10:36:48.614854 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0929 10:36:48.614907 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
W20260929 10:36:48.615161 32627 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260929 10:36:48.617883 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 32498
I20260929 10:36:49.254736 32631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:49.254208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:44709 (local address 127.15.82.65:46687)
0929 10:36:49.254548 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0929 10:36:49.254615 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":78,"threads_started":1}
I20260929 10:36:49.415529 32632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:49.415125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:44709 (local address 127.15.82.66:43415)
0929 10:36:49.415372 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0929 10:36:49.415427 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":69,"threads_started":1}
I20260929 10:36:49.616114 32633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:49.615682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:44709 (local address 127.15.82.67:55169)
0929 10:36:49.615957 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0929 10:36:49.616013 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":102,"threads_started":1}
I20260929 10:36:49.718163 32634 raft_consensus.cc:482] T e757f25757a940fba2a32c51bc0ac3bf P 63f8e03d49954851a2b4b48305144ee5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:36:49.718497 32634 raft_consensus.cc:504] T e757f25757a940fba2a32c51bc0ac3bf P 63f8e03d49954851a2b4b48305144ee5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6581ebffa3246a987710e539ce43fa3" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 35159 } } peers { permanent_uuid: "04ba93cc78414e468e3bb02139bc71c0" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 44377 } } peers { permanent_uuid: "63f8e03d49954851a2b4b48305144ee5" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 36197 } }
I20260929 10:36:49.719632 32634 leader_election.cc:288] T e757f25757a940fba2a32c51bc0ac3bf P 63f8e03d49954851a2b4b48305144ee5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e6581ebffa3246a987710e539ce43fa3 (127.15.82.66:35159), 04ba93cc78414e468e3bb02139bc71c0 (127.15.82.67:44377)
Sep 29 10:36:49 dist-test-slave-vg89 krb5kdc[31052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678208, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.65@KRBTEST.COM for kudu/127.15.82.67@KRBTEST.COM
Sep 29 10:36:49 dist-test-slave-vg89 krb5kdc[31052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678208, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.65@KRBTEST.COM for kudu/127.15.82.66@KRBTEST.COM
I20260929 10:36:49.734735 32636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:49.720143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.67:44377 (local address 127.15.82.65:36921)
0929 10:36:49.720438 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0929 10:36:49.720453 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 10:36:49.720547 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:36:49.721998 (+ 1451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:36:49.722001 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:36:49.722557 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:36:49.723017 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:36:49.723031 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:36:49.724304 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:36:49.724311 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:36:49.725292 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:36:49.725301 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:36:49.726111 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:36:49.726114 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:36:49.726372 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:36:49.727238 (+ 866us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:36:49.727258 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:36:49.729014 (+ 1756us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:36:49.731752 (+ 2738us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:36:49.731757 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:36:49.731769 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:36:49.732077 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:36:49.732470 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:36:49.732472 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:36:49.732473 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:36:49.732630 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:36:49.733539 (+ 909us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:36:49.733546 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:36:49.733928 (+ 382us) client_negotiation.cc:769] Sending connection context
0929 10:36:49.734150 (+ 222us) client_negotiation.cc:240] Negotiation successful
0929 10:36:49.734583 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"mutex_wait_us":348,"spinlock_wait_cycles":32000,"thread_start_us":76,"threads_started":1}
I20260929 10:36:49.734863 32635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:49.720046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.65:36505 (local address 127.15.82.66:35159)
0929 10:36:49.720466 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0929 10:36:49.720488 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0929 10:36:49.720526 (+ 38us) server_negotiation.cc:407] Connection header received
0929 10:36:49.721243 (+ 717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:36:49.721250 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:36:49.721606 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:36:49.721875 (+ 269us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:36:49.723194 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:36:49.724264 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:36:49.725464 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:36:49.725942 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:36:49.726107 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:36:49.729688 (+ 3581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:36:49.729698 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:36:49.729726 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:36:49.729759 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:36:49.731933 (+ 2174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:36:49.732378 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:36:49.732384 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:36:49.732389 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:36:49.732538 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:36:49.732843 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:36:49.732849 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:36:49.732854 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:36:49.733403 (+ 549us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:36:49.733555 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0929 10:36:49.734315 (+ 760us) server_negotiation.cc:299] Negotiation successful
0929 10:36:49.734670 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":107,"threads_started":1}
I20260929 10:36:49.735134 32637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:49.720329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.65:36921 (local address 127.15.82.67:44377)
0929 10:36:49.720744 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0929 10:36:49.720763 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0929 10:36:49.720793 (+ 30us) server_negotiation.cc:407] Connection header received
0929 10:36:49.721424 (+ 631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:36:49.721429 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:36:49.721703 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:36:49.721999 (+ 296us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:36:49.723195 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:36:49.724150 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:36:49.725442 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:36:49.725954 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:36:49.726150 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:36:49.729287 (+ 3137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:36:49.729295 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:36:49.729313 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:36:49.729361 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:36:49.731619 (+ 2258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:36:49.732197 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:36:49.732202 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:36:49.732207 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:36:49.732368 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:36:49.732763 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:36:49.732807 (+ 44us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:36:49.732811 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:36:49.733403 (+ 592us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:36:49.733555 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0929 10:36:49.734631 (+ 1076us) server_negotiation.cc:299] Negotiation successful
0929 10:36:49.734932 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":115,"threads_started":1}
I20260929 10:36:49.734737 32631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:49.719871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.66:35159 (local address 127.15.82.65:36505)
0929 10:36:49.720023 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0929 10:36:49.720041 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 10:36:49.720171 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:36:49.721983 (+ 1812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:36:49.721989 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:36:49.722548 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:36:49.723017 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:36:49.723031 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:36:49.724425 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:36:49.724428 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:36:49.725291 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:36:49.725300 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:36:49.726075 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:36:49.726079 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:36:49.726374 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:36:49.727183 (+ 809us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:36:49.727200 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:36:49.729485 (+ 2285us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:36:49.732048 (+ 2563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:36:49.732051 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:36:49.732053 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:36:49.732274 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:36:49.732629 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:36:49.732632 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:36:49.732634 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:36:49.732729 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:36:49.733539 (+ 810us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:36:49.733546 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:36:49.733928 (+ 382us) client_negotiation.cc:769] Sending connection context
0929 10:36:49.734150 (+ 222us) client_negotiation.cc:240] Negotiation successful
0929 10:36:49.734527 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61,"mutex_wait_us":251,"spinlock_wait_cycles":8320}
W20260929 10:36:49.737730 32250 leader_election.cc:341] T e757f25757a940fba2a32c51bc0ac3bf P 63f8e03d49954851a2b4b48305144ee5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e6581ebffa3246a987710e539ce43fa3 (127.15.82.66:35159): Not found: Tablet not found: e757f25757a940fba2a32c51bc0ac3bf
W20260929 10:36:49.737979 32250 leader_election.cc:341] T e757f25757a940fba2a32c51bc0ac3bf P 63f8e03d49954851a2b4b48305144ee5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 04ba93cc78414e468e3bb02139bc71c0 (127.15.82.67:44377): Not found: Tablet not found: e757f25757a940fba2a32c51bc0ac3bf
I20260929 10:36:49.738052 32250 leader_election.cc:302] T e757f25757a940fba2a32c51bc0ac3bf P 63f8e03d49954851a2b4b48305144ee5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 63f8e03d49954851a2b4b48305144ee5; no voters: 04ba93cc78414e468e3bb02139bc71c0, e6581ebffa3246a987710e539ce43fa3
I20260929 10:36:49.738349 32634 raft_consensus.cc:2726] T e757f25757a940fba2a32c51bc0ac3bf P 63f8e03d49954851a2b4b48305144ee5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 10:36:50.256039 32638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:50.255538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:44709 (local address 127.15.82.65:48335)
0929 10:36:50.255825 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0929 10:36:50.255903 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":72,"threads_started":1}
I20260929 10:36:50.416779 32639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:50.416386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:44709 (local address 127.15.82.66:53527)
0929 10:36:50.416636 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0929 10:36:50.416693 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":69,"threads_started":1}
I20260929 10:36:50.617403 32640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:50.616929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:44709 (local address 127.15.82.67:41609)
0929 10:36:50.617211 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0929 10:36:50.617303 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":85,"threads_started":1}
I20260929 10:36:51.257288 32641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:51.256823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:44709 (local address 127.15.82.65:57139)
0929 10:36:51.257137 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0929 10:36:51.257192 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":126,"threads_started":1}
I20260929 10:36:51.282260 32642 raft_consensus.cc:482] T e757f25757a940fba2a32c51bc0ac3bf P 63f8e03d49954851a2b4b48305144ee5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:36:51.282442 32642 raft_consensus.cc:504] T e757f25757a940fba2a32c51bc0ac3bf P 63f8e03d49954851a2b4b48305144ee5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6581ebffa3246a987710e539ce43fa3" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 35159 } } peers { permanent_uuid: "04ba93cc78414e468e3bb02139bc71c0" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 44377 } } peers { permanent_uuid: "63f8e03d49954851a2b4b48305144ee5" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 36197 } }
I20260929 10:36:51.282934 32642 leader_election.cc:288] T e757f25757a940fba2a32c51bc0ac3bf P 63f8e03d49954851a2b4b48305144ee5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e6581ebffa3246a987710e539ce43fa3 (127.15.82.66:35159), 04ba93cc78414e468e3bb02139bc71c0 (127.15.82.67:44377)
W20260929 10:36:51.283855 32250 leader_election.cc:341] T e757f25757a940fba2a32c51bc0ac3bf P 63f8e03d49954851a2b4b48305144ee5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 04ba93cc78414e468e3bb02139bc71c0 (127.15.82.67:44377): Not found: Tablet not found: e757f25757a940fba2a32c51bc0ac3bf
W20260929 10:36:51.284015 32250 leader_election.cc:341] T e757f25757a940fba2a32c51bc0ac3bf P 63f8e03d49954851a2b4b48305144ee5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e6581ebffa3246a987710e539ce43fa3 (127.15.82.66:35159): Not found: Tablet not found: e757f25757a940fba2a32c51bc0ac3bf
I20260929 10:36:51.284080 32250 leader_election.cc:302] T e757f25757a940fba2a32c51bc0ac3bf P 63f8e03d49954851a2b4b48305144ee5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 63f8e03d49954851a2b4b48305144ee5; no voters: 04ba93cc78414e468e3bb02139bc71c0, e6581ebffa3246a987710e539ce43fa3
I20260929 10:36:51.284260 32642 raft_consensus.cc:2726] T e757f25757a940fba2a32c51bc0ac3bf P 63f8e03d49954851a2b4b48305144ee5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 10:36:51.418095 32643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:51.417529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:44709 (local address 127.15.82.66:56575)
0929 10:36:51.417867 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0929 10:36:51.417949 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":89,"threads_started":1}
I20260929 10:36:51.618573 32644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:51.618098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:44709 (local address 127.15.82.67:56227)
0929 10:36:51.618341 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0929 10:36:51.618452 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":74,"threads_started":1}
I20260929 10:36:52.258735 32645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:52.258214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:44709 (local address 127.15.82.65:43873)
0929 10:36:52.258551 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0929 10:36:52.258614 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":84,"threads_started":1}
I20260929 10:36:52.419332 32646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:52.418940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:44709 (local address 127.15.82.66:55353)
0929 10:36:52.419179 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0929 10:36:52.419237 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":74,"threads_started":1}
I20260929 10:36:52.619669 32647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:52.619280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:44709 (local address 127.15.82.67:51799)
0929 10:36:52.619523 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0929 10:36:52.619577 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":88,"threads_started":1}
I20260929 10:36:53.260200 32648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:53.259693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:44709 (local address 127.15.82.65:34883)
0929 10:36:53.260043 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0929 10:36:53.260097 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":108,"threads_started":1}
I20260929 10:36:53.267634 32649 raft_consensus.cc:482] T e757f25757a940fba2a32c51bc0ac3bf P 63f8e03d49954851a2b4b48305144ee5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:36:53.267747 32649 raft_consensus.cc:504] T e757f25757a940fba2a32c51bc0ac3bf P 63f8e03d49954851a2b4b48305144ee5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6581ebffa3246a987710e539ce43fa3" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 35159 } } peers { permanent_uuid: "04ba93cc78414e468e3bb02139bc71c0" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 44377 } } peers { permanent_uuid: "63f8e03d49954851a2b4b48305144ee5" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 36197 } }
I20260929 10:36:53.268144 32649 leader_election.cc:288] T e757f25757a940fba2a32c51bc0ac3bf P 63f8e03d49954851a2b4b48305144ee5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e6581ebffa3246a987710e539ce43fa3 (127.15.82.66:35159), 04ba93cc78414e468e3bb02139bc71c0 (127.15.82.67:44377)
W20260929 10:36:53.268879 32250 leader_election.cc:341] T e757f25757a940fba2a32c51bc0ac3bf P 63f8e03d49954851a2b4b48305144ee5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e6581ebffa3246a987710e539ce43fa3 (127.15.82.66:35159): Not found: Tablet not found: e757f25757a940fba2a32c51bc0ac3bf
W20260929 10:36:53.269038 32250 leader_election.cc:341] T e757f25757a940fba2a32c51bc0ac3bf P 63f8e03d49954851a2b4b48305144ee5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 04ba93cc78414e468e3bb02139bc71c0 (127.15.82.67:44377): Not found: Tablet not found: e757f25757a940fba2a32c51bc0ac3bf
I20260929 10:36:53.269109 32250 leader_election.cc:302] T e757f25757a940fba2a32c51bc0ac3bf P 63f8e03d49954851a2b4b48305144ee5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 63f8e03d49954851a2b4b48305144ee5; no voters: 04ba93cc78414e468e3bb02139bc71c0, e6581ebffa3246a987710e539ce43fa3
I20260929 10:36:53.269274 32649 raft_consensus.cc:2726] T e757f25757a940fba2a32c51bc0ac3bf P 63f8e03d49954851a2b4b48305144ee5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 10:36:53.420514 32650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:53.420104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:44709 (local address 127.15.82.66:43333)
0929 10:36:53.420357 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0929 10:36:53.420426 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":88,"threads_started":1}
I20260929 10:36:53.620990 32651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:53.620576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:44709 (local address 127.15.82.67:46339)
0929 10:36:53.620840 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0929 10:36:53.620897 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":88,"threads_started":1}
I20260929 10:36:54.261386 32652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:54.260918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:44709 (local address 127.15.82.65:46269)
0929 10:36:54.261163 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0929 10:36:54.261250 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":72,"threads_started":1}
I20260929 10:36:54.421625 32653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:54.421222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:44709 (local address 127.15.82.66:53215)
0929 10:36:54.421484 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0929 10:36:54.421534 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":74,"threads_started":1}
I20260929 10:36:54.622179 32654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:54.621740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:44709 (local address 127.15.82.67:39559)
0929 10:36:54.622016 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0929 10:36:54.622074 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":80,"threads_started":1}
I20260929 10:36:55.262732 32655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:55.262192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:44709 (local address 127.15.82.65:51455)
0929 10:36:55.262544 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0929 10:36:55.262600 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":95,"threads_started":1}
I20260929 10:36:55.422932 32656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:55.422479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:44709 (local address 127.15.82.66:52133)
0929 10:36:55.422769 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0929 10:36:55.422828 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":73,"threads_started":1}
I20260929 10:36:55.524358 32657 raft_consensus.cc:482] T e757f25757a940fba2a32c51bc0ac3bf P 63f8e03d49954851a2b4b48305144ee5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:36:55.524498 32657 raft_consensus.cc:504] T e757f25757a940fba2a32c51bc0ac3bf P 63f8e03d49954851a2b4b48305144ee5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6581ebffa3246a987710e539ce43fa3" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 35159 } } peers { permanent_uuid: "04ba93cc78414e468e3bb02139bc71c0" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 44377 } } peers { permanent_uuid: "63f8e03d49954851a2b4b48305144ee5" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 36197 } }
I20260929 10:36:55.525053 32657 leader_election.cc:288] T e757f25757a940fba2a32c51bc0ac3bf P 63f8e03d49954851a2b4b48305144ee5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e6581ebffa3246a987710e539ce43fa3 (127.15.82.66:35159), 04ba93cc78414e468e3bb02139bc71c0 (127.15.82.67:44377)
W20260929 10:36:55.525889 32250 leader_election.cc:341] T e757f25757a940fba2a32c51bc0ac3bf P 63f8e03d49954851a2b4b48305144ee5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 04ba93cc78414e468e3bb02139bc71c0 (127.15.82.67:44377): Not found: Tablet not found: e757f25757a940fba2a32c51bc0ac3bf
W20260929 10:36:55.526054 32250 leader_election.cc:341] T e757f25757a940fba2a32c51bc0ac3bf P 63f8e03d49954851a2b4b48305144ee5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e6581ebffa3246a987710e539ce43fa3 (127.15.82.66:35159): Not found: Tablet not found: e757f25757a940fba2a32c51bc0ac3bf
I20260929 10:36:55.526140 32250 leader_election.cc:302] T e757f25757a940fba2a32c51bc0ac3bf P 63f8e03d49954851a2b4b48305144ee5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 63f8e03d49954851a2b4b48305144ee5; no voters: 04ba93cc78414e468e3bb02139bc71c0, e6581ebffa3246a987710e539ce43fa3
I20260929 10:36:55.526363 32657 raft_consensus.cc:2726] T e757f25757a940fba2a32c51bc0ac3bf P 63f8e03d49954851a2b4b48305144ee5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 10:36:55.623854 32658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:55.623177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:44709 (local address 127.15.82.67:53305)
0929 10:36:55.623573 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0929 10:36:55.623652 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":99,"threads_started":1}
I20260929 10:36:56.263906 32659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:56.263448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:44709 (local address 127.15.82.65:39517)
0929 10:36:56.263716 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0929 10:36:56.263810 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":76,"threads_started":1}
I20260929 10:36:56.424360 32660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:56.423835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:44709 (local address 127.15.82.66:45489)
0929 10:36:56.424162 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0929 10:36:56.424219 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":85,"threads_started":1}
I20260929 10:36:56.625104 32661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:56.624673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:44709 (local address 127.15.82.67:58951)
0929 10:36:56.624944 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0929 10:36:56.625004 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":76,"threads_started":1}
I20260929 10:36:57.265147 32662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:57.264689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:44709 (local address 127.15.82.65:54687)
0929 10:36:57.264983 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0929 10:36:57.265048 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":80,"threads_started":1}
I20260929 10:36:57.425700 32663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:57.425236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:44709 (local address 127.15.82.66:41615)
0929 10:36:57.425530 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0929 10:36:57.425599 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":79,"threads_started":1}
I20260929 10:36:57.626822 32664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:57.626182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:44709 (local address 127.15.82.67:53085)
0929 10:36:57.626657 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0929 10:36:57.626714 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":155,"threads_started":1}
I20260929 10:36:57.782555 32665 raft_consensus.cc:482] T e757f25757a940fba2a32c51bc0ac3bf P 63f8e03d49954851a2b4b48305144ee5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:36:57.782712 32665 raft_consensus.cc:504] T e757f25757a940fba2a32c51bc0ac3bf P 63f8e03d49954851a2b4b48305144ee5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6581ebffa3246a987710e539ce43fa3" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 35159 } } peers { permanent_uuid: "04ba93cc78414e468e3bb02139bc71c0" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 44377 } } peers { permanent_uuid: "63f8e03d49954851a2b4b48305144ee5" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 36197 } }
I20260929 10:36:57.783119 32665 leader_election.cc:288] T e757f25757a940fba2a32c51bc0ac3bf P 63f8e03d49954851a2b4b48305144ee5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e6581ebffa3246a987710e539ce43fa3 (127.15.82.66:35159), 04ba93cc78414e468e3bb02139bc71c0 (127.15.82.67:44377)
W20260929 10:36:57.783970 32250 leader_election.cc:341] T e757f25757a940fba2a32c51bc0ac3bf P 63f8e03d49954851a2b4b48305144ee5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e6581ebffa3246a987710e539ce43fa3 (127.15.82.66:35159): Not found: Tablet not found: e757f25757a940fba2a32c51bc0ac3bf
W20260929 10:36:57.784132 32250 leader_election.cc:341] T e757f25757a940fba2a32c51bc0ac3bf P 63f8e03d49954851a2b4b48305144ee5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 04ba93cc78414e468e3bb02139bc71c0 (127.15.82.67:44377): Not found: Tablet not found: e757f25757a940fba2a32c51bc0ac3bf
I20260929 10:36:57.784192 32250 leader_election.cc:302] T e757f25757a940fba2a32c51bc0ac3bf P 63f8e03d49954851a2b4b48305144ee5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 63f8e03d49954851a2b4b48305144ee5; no voters: 04ba93cc78414e468e3bb02139bc71c0, e6581ebffa3246a987710e539ce43fa3
I20260929 10:36:57.784379 32665 raft_consensus.cc:2726] T e757f25757a940fba2a32c51bc0ac3bf P 63f8e03d49954851a2b4b48305144ee5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 10:36:58.266487 32666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:58.266005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:44709 (local address 127.15.82.65:52995)
0929 10:36:58.266273 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0929 10:36:58.266330 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":73,"threads_started":1}
I20260929 10:36:58.426998 32667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:58.426596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:44709 (local address 127.15.82.66:56751)
0929 10:36:58.426846 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0929 10:36:58.426907 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":77,"threads_started":1}
I20260929 10:36:58.627995 32668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:58.627580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:44709 (local address 127.15.82.67:57623)
0929 10:36:58.627843 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0929 10:36:58.627901 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":87,"threads_started":1}
I20260929 10:36:59.267681 32669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:59.267264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:44709 (local address 127.15.82.65:44679)
0929 10:36:59.267526 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0929 10:36:59.267585 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":88,"threads_started":1}
I20260929 10:36:59.428244 32670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:59.427848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:44709 (local address 127.15.82.66:53575)
0929 10:36:59.428094 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0929 10:36:59.428149 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":74,"threads_started":1}
I20260929 10:36:59.629482 32671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:36:59.628886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:44709 (local address 127.15.82.67:58765)
0929 10:36:59.629297 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0929 10:36:59.629368 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":124,"threads_started":1}
I20260929 10:37:00.225864 32672 raft_consensus.cc:482] T e757f25757a940fba2a32c51bc0ac3bf P 63f8e03d49954851a2b4b48305144ee5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:37:00.226006 32672 raft_consensus.cc:504] T e757f25757a940fba2a32c51bc0ac3bf P 63f8e03d49954851a2b4b48305144ee5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6581ebffa3246a987710e539ce43fa3" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 35159 } } peers { permanent_uuid: "04ba93cc78414e468e3bb02139bc71c0" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 44377 } } peers { permanent_uuid: "63f8e03d49954851a2b4b48305144ee5" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 36197 } }
I20260929 10:37:00.226362 32672 leader_election.cc:288] T e757f25757a940fba2a32c51bc0ac3bf P 63f8e03d49954851a2b4b48305144ee5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e6581ebffa3246a987710e539ce43fa3 (127.15.82.66:35159), 04ba93cc78414e468e3bb02139bc71c0 (127.15.82.67:44377)
W20260929 10:37:00.227196 32250 leader_election.cc:341] T e757f25757a940fba2a32c51bc0ac3bf P 63f8e03d49954851a2b4b48305144ee5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e6581ebffa3246a987710e539ce43fa3 (127.15.82.66:35159): Not found: Tablet not found: e757f25757a940fba2a32c51bc0ac3bf
W20260929 10:37:00.227391 32250 leader_election.cc:341] T e757f25757a940fba2a32c51bc0ac3bf P 63f8e03d49954851a2b4b48305144ee5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 04ba93cc78414e468e3bb02139bc71c0 (127.15.82.67:44377): Not found: Tablet not found: e757f25757a940fba2a32c51bc0ac3bf
I20260929 10:37:00.227449 32250 leader_election.cc:302] T e757f25757a940fba2a32c51bc0ac3bf P 63f8e03d49954851a2b4b48305144ee5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 63f8e03d49954851a2b4b48305144ee5; no voters: 04ba93cc78414e468e3bb02139bc71c0, e6581ebffa3246a987710e539ce43fa3
I20260929 10:37:00.227581 32672 raft_consensus.cc:2726] T e757f25757a940fba2a32c51bc0ac3bf P 63f8e03d49954851a2b4b48305144ee5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 10:37:00.269095 32673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:00.268595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:44709 (local address 127.15.82.65:55839)
0929 10:37:00.268933 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:00.268997 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":95,"threads_started":1}
I20260929 10:37:00.429476 32674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:00.429032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:44709 (local address 127.15.82.66:56659)
0929 10:37:00.429325 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:00.429381 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":89,"threads_started":1}
I20260929 10:37:00.630884 32675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:00.630345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:44709 (local address 127.15.82.67:52801)
0929 10:37:00.630671 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:00.630733 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":156,"threads_started":1}
I20260929 10:37:01.270292 32676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:01.269817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:44709 (local address 127.15.82.65:39393)
0929 10:37:01.270141 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:01.270197 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":70,"threads_started":1}
I20260929 10:37:01.430689 32677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:01.430219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:44709 (local address 127.15.82.66:51755)
0929 10:37:01.430536 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:01.430592 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":73,"threads_started":1}
I20260929 10:37:01.632161 32678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:01.631703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:44709 (local address 127.15.82.67:55487)
0929 10:37:01.631987 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:01.632050 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":79,"threads_started":1}
I20260929 10:37:02.271754 32679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:02.271301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:44709 (local address 127.15.82.65:33735)
0929 10:37:02.271583 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:02.271651 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":87,"threads_started":1}
I20260929 10:37:02.431912 32680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:02.431510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:44709 (local address 127.15.82.66:41977)
0929 10:37:02.431757 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:02.431813 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":79,"threads_started":1}
I20260929 10:37:02.633481 32681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:02.633012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:44709 (local address 127.15.82.67:40167)
0929 10:37:02.633317 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:02.633380 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":89,"threads_started":1}
I20260929 10:37:02.860769 32682 raft_consensus.cc:482] T e757f25757a940fba2a32c51bc0ac3bf P 63f8e03d49954851a2b4b48305144ee5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:37:02.860924 32682 raft_consensus.cc:504] T e757f25757a940fba2a32c51bc0ac3bf P 63f8e03d49954851a2b4b48305144ee5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6581ebffa3246a987710e539ce43fa3" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 35159 } } peers { permanent_uuid: "04ba93cc78414e468e3bb02139bc71c0" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 44377 } } peers { permanent_uuid: "63f8e03d49954851a2b4b48305144ee5" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 36197 } }
I20260929 10:37:02.861338 32682 leader_election.cc:288] T e757f25757a940fba2a32c51bc0ac3bf P 63f8e03d49954851a2b4b48305144ee5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e6581ebffa3246a987710e539ce43fa3 (127.15.82.66:35159), 04ba93cc78414e468e3bb02139bc71c0 (127.15.82.67:44377)
W20260929 10:37:02.862121 32250 leader_election.cc:341] T e757f25757a940fba2a32c51bc0ac3bf P 63f8e03d49954851a2b4b48305144ee5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e6581ebffa3246a987710e539ce43fa3 (127.15.82.66:35159): Not found: Tablet not found: e757f25757a940fba2a32c51bc0ac3bf
W20260929 10:37:02.862252 32250 leader_election.cc:341] T e757f25757a940fba2a32c51bc0ac3bf P 63f8e03d49954851a2b4b48305144ee5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 04ba93cc78414e468e3bb02139bc71c0 (127.15.82.67:44377): Not found: Tablet not found: e757f25757a940fba2a32c51bc0ac3bf
I20260929 10:37:02.862289 32250 leader_election.cc:302] T e757f25757a940fba2a32c51bc0ac3bf P 63f8e03d49954851a2b4b48305144ee5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 63f8e03d49954851a2b4b48305144ee5; no voters: 04ba93cc78414e468e3bb02139bc71c0, e6581ebffa3246a987710e539ce43fa3
I20260929 10:37:02.862457 32682 raft_consensus.cc:2726] T e757f25757a940fba2a32c51bc0ac3bf P 63f8e03d49954851a2b4b48305144ee5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 10:37:03.273386 32683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:03.272902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:44709 (local address 127.15.82.65:55649)
0929 10:37:03.273203 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:03.273283 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":90,"threads_started":1}
I20260929 10:37:03.433139 32684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:03.432768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:44709 (local address 127.15.82.66:55341)
0929 10:37:03.432992 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:03.433046 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":70,"threads_started":1}
I20260929 10:37:03.625239 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 32198
I20260929 10:37:03.631282 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 32364
I20260929 10:37:03.635031 32685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:03.634588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:44709 (local address 127.15.82.67:59261)
0929 10:37:03.634871 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:03.634926 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.126:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":83,"threads_started":1}
I20260929 10:37:03.637342 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 32498
2026-09-29T10:37:03Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59454 ms)
[----------] 30 tests from SecurityITest (242034 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:37:03 dist-test-slave-vg89 krb5kdc[32688](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:37:03 dist-test-slave-vg89 krb5kdc[32688](info): set up 2 sockets
Sep 29 10:37:03 dist-test-slave-vg89 krb5kdc[32688](info): commencing operation
krb5kdc: starting...
W20260929 10:37:04.674646 15689 mini_kdc.cc:121] Time spent starting KDC: real 1.014s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Sep 29 10:37:04 dist-test-slave-vg89 krb5kdc[32688](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678224, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:37:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:37:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.82.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab.
Entry for principal kudu/127.15.82.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab.
I20260929 10:37:04.801838 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:39465
--webserver_interface=127.15.82.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab
--principal=kudu/127.15.82.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:38831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.82.126:39465
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:37:04.914297 32701 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:37:04.914642 32701 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:37:04.914687 32701 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:37:04.918792 32701 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:37:04.918849 32701 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:37:04.918869 32701 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:37:04.918901 32701 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:37:04.918934 32701 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:37:04.924278 32701 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:38831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.82.126:39465
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab
--principal=kudu/127.15.82.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:39465
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.82.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32701
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:37:04.925380 32701 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:37:04.926440 32701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 10:37:04.935086 32701 server_base.cc:1034] running on GCE node
W20260929 10:37:04.935230 32709 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:04.935117 32706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:04.935133 32707 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:37:04.935917 32701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:37:04.936995 32701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:37:04.938154 32701 hybrid_clock.cc:648] HybridClock initialized: now 1790678224938098 us; error 68 us; skew 500 ppm
Sep 29 10:37:04 dist-test-slave-vg89 krb5kdc[32688](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678224, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:37:04.940872 32701 init.cc:377] Logged in from keytab as kudu/127.15.82.126@KRBTEST.COM (short username kudu)
I20260929 10:37:04.941716 32701 webserver.cc:460] Webserver started at http://127.15.82.126:36739/ using document root <none> and password file <none>
I20260929 10:37:04.942269 32701 fs_manager.cc:359] Metadata directory not provided
I20260929 10:37:04.942368 32701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:37:04.942611 32701 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:37:04.944578 32701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/minicluster-data/master-0/data/instance:
uuid: "0794cecd6ef9491892ee1c23f39d317a"
format_stamp: "Formatted at 2026-09-29 10:37:04 on dist-test-slave-vg89"
server_key: "908e11c0d1ee7c97b1fe175fa88dd272"
server_key_iv: "242bcab29be17c821c9a5bff87160b7b"
server_key_version: "encryptionkey@0"
I20260929 10:37:04.945124 32701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/minicluster-data/master-0/wal/instance:
uuid: "0794cecd6ef9491892ee1c23f39d317a"
format_stamp: "Formatted at 2026-09-29 10:37:04 on dist-test-slave-vg89"
server_key: "908e11c0d1ee7c97b1fe175fa88dd272"
server_key_iv: "242bcab29be17c821c9a5bff87160b7b"
server_key_version: "encryptionkey@0"
I20260929 10:37:04.948797 32701 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.005s
I20260929 10:37:04.951179 32716 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:04.952406 32701 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.002s
I20260929 10:37:04.952562 32701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/minicluster-data/master-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/minicluster-data/master-0/wal
uuid: "0794cecd6ef9491892ee1c23f39d317a"
format_stamp: "Formatted at 2026-09-29 10:37:04 on dist-test-slave-vg89"
server_key: "908e11c0d1ee7c97b1fe175fa88dd272"
server_key_iv: "242bcab29be17c821c9a5bff87160b7b"
server_key_version: "encryptionkey@0"
I20260929 10:37:04.952677 32701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:37:04.963910 32701 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:37:04.967098 32701 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:37:04.967311 32701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:37:04.974143 32701 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.126:39465
I20260929 10:37:04.974205 300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.126:39465 every 8 connection(s)
I20260929 10:37:04.975162 32701 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
I20260929 10:37:04.977952 301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:37:04.978144 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 32701
I20260929 10:37:04.978277 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/minicluster-data/master-0/wal/instance
I20260929 10:37:04.978634 15689 external_mini_cluster.cc:1428] Setting key baa43beafbc456bd9bd43d7582a7f858
I20260929 10:37:04.985131 301 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0794cecd6ef9491892ee1c23f39d317a: Bootstrap starting.
I20260929 10:37:04.987608 301 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0794cecd6ef9491892ee1c23f39d317a: Neither blocks nor log segments found. Creating new log.
Sep 29 10:37:04 dist-test-slave-vg89 krb5kdc[32688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678224, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:37:04.988497 301 log.cc:824] T 00000000000000000000000000000000 P 0794cecd6ef9491892ee1c23f39d317a: Log is configured to *not* fsync() on all Append() calls
I20260929 10:37:04.990373 301 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0794cecd6ef9491892ee1c23f39d317a: No bootstrap required, opened a new log
I20260929 10:37:04.993628 304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:04.980057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53062 (local address 127.15.82.126:39465)
0929 10:37:04.980463 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:04.980482 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:04.980525 (+ 43us) server_negotiation.cc:407] Connection header received
0929 10:37:04.981374 (+ 849us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:04.981398 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:04.981829 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:04.982198 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:04.982970 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:04.984502 (+ 1532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:04.985423 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:04.986072 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:04.986264 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:04.989167 (+ 2903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:04.989172 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:04.989185 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:37:04.989210 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:04.990979 (+ 1769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:04.991414 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:04.991420 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:04.991430 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:04.991535 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:04.991809 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:04.991812 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:04.991813 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:04.992189 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:04.992377 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:04.992668 (+ 291us) server_negotiation.cc:299] Negotiation successful
0929 10:37:04.992950 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":109,"threads_started":1}
I20260929 10:37:04.994001 301 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0794cecd6ef9491892ee1c23f39d317a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0794cecd6ef9491892ee1c23f39d317a" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 39465 } }
I20260929 10:37:04.994235 301 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0794cecd6ef9491892ee1c23f39d317a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:37:04.994318 301 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0794cecd6ef9491892ee1c23f39d317a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0794cecd6ef9491892ee1c23f39d317a, State: Initialized, Role: FOLLOWER
I20260929 10:37:04.994755 301 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0794cecd6ef9491892ee1c23f39d317a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0794cecd6ef9491892ee1c23f39d317a" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 39465 } }
I20260929 10:37:04.994917 301 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0794cecd6ef9491892ee1c23f39d317a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:37:04.994998 301 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0794cecd6ef9491892ee1c23f39d317a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:37:04.995107 301 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0794cecd6ef9491892ee1c23f39d317a [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:37:04.996084 301 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0794cecd6ef9491892ee1c23f39d317a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0794cecd6ef9491892ee1c23f39d317a" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 39465 } }
I20260929 10:37:04.996410 301 leader_election.cc:302] T 00000000000000000000000000000000 P 0794cecd6ef9491892ee1c23f39d317a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0794cecd6ef9491892ee1c23f39d317a; no voters:
I20260929 10:37:04.996706 301 leader_election.cc:288] T 00000000000000000000000000000000 P 0794cecd6ef9491892ee1c23f39d317a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:37:04.996846 306 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0794cecd6ef9491892ee1c23f39d317a [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:37:04.997107 306 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0794cecd6ef9491892ee1c23f39d317a [term 1 LEADER]: Becoming Leader. State: Replica: 0794cecd6ef9491892ee1c23f39d317a, State: Running, Role: LEADER
I20260929 10:37:04.997421 306 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0794cecd6ef9491892ee1c23f39d317a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0794cecd6ef9491892ee1c23f39d317a" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 39465 } }
I20260929 10:37:04.997859 301 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0794cecd6ef9491892ee1c23f39d317a [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:37:04.999153 308 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0794cecd6ef9491892ee1c23f39d317a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0794cecd6ef9491892ee1c23f39d317a. Latest consensus state: current_term: 1 leader_uuid: "0794cecd6ef9491892ee1c23f39d317a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0794cecd6ef9491892ee1c23f39d317a" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 39465 } } }
I20260929 10:37:04.999161 307 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0794cecd6ef9491892ee1c23f39d317a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0794cecd6ef9491892ee1c23f39d317a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0794cecd6ef9491892ee1c23f39d317a" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 39465 } } }
I20260929 10:37:04.999327 308 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0794cecd6ef9491892ee1c23f39d317a [sys.catalog]: This master's current role is: LEADER
I20260929 10:37:04.999333 307 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0794cecd6ef9491892ee1c23f39d317a [sys.catalog]: This master's current role is: LEADER
I20260929 10:37:05.000373 315 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:37:05.003286 315 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:37:05.008764 315 catalog_manager.cc:1347] Generated new cluster ID: c53d0a8caa8044019cc7da637bdf95eb
I20260929 10:37:05.008829 315 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:37:05.024165 315 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:37:05.025480 315 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:37:05.032915 315 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0794cecd6ef9491892ee1c23f39d317a: Generated new TSK 0
I20260929 10:37:05.033746 315 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.82.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab.
Entry for principal kudu/127.15.82.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab.
I20260929 10:37:05.069077 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.65:0
--local_ip_for_outbound_sockets=127.15.82.65
--webserver_interface=127.15.82.65
--webserver_port=0
--tserver_master_addrs=127.15.82.126:39465
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab
--principal=kudu/127.15.82.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:38831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:37:05.184530 327 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:37:05.184877 327 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:37:05.184948 327 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:37:05.189311 327 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:37:05.189385 327 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:37:05.189467 327 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.65
I20260929 10:37:05.195235 327 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:38831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab
--principal=kudu/127.15.82.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.82.65
--webserver_port=0
--tserver_master_addrs=127.15.82.126:39465
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.327
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.65
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:37:05.196477 327 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:37:05.197517 327 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:37:05.207533 333 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:05.207551 332 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:37:05.207595 327 server_base.cc:1034] running on GCE node
W20260929 10:37:05.207793 335 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:37:05.208175 327 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:37:05.208807 327 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:37:05.209985 327 hybrid_clock.cc:648] HybridClock initialized: now 1790678225209970 us; error 28 us; skew 500 ppm
Sep 29 10:37:05 dist-test-slave-vg89 krb5kdc[32688](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678225, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:37:05.212818 327 init.cc:377] Logged in from keytab as kudu/127.15.82.65@KRBTEST.COM (short username kudu)
I20260929 10:37:05.213672 327 webserver.cc:460] Webserver started at http://127.15.82.65:37237/ using document root <none> and password file <none>
I20260929 10:37:05.214232 327 fs_manager.cc:359] Metadata directory not provided
I20260929 10:37:05.214324 327 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:37:05.214577 327 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:37:05.216642 327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/minicluster-data/ts-0/data/instance:
uuid: "86888832d7b541839e81ac540dcbfa74"
format_stamp: "Formatted at 2026-09-29 10:37:05 on dist-test-slave-vg89"
server_key: "6582a7649f35168cba1f2646a4ab55c7"
server_key_iv: "c7842b1eafe57b32b2dc4221c4a70679"
server_key_version: "encryptionkey@0"
I20260929 10:37:05.217207 327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/minicluster-data/ts-0/wal/instance:
uuid: "86888832d7b541839e81ac540dcbfa74"
format_stamp: "Formatted at 2026-09-29 10:37:05 on dist-test-slave-vg89"
server_key: "6582a7649f35168cba1f2646a4ab55c7"
server_key_iv: "c7842b1eafe57b32b2dc4221c4a70679"
server_key_version: "encryptionkey@0"
I20260929 10:37:05.220898 327 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260929 10:37:05.223341 342 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:05.224587 327 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260929 10:37:05.224741 327 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/minicluster-data/ts-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/minicluster-data/ts-0/wal
uuid: "86888832d7b541839e81ac540dcbfa74"
format_stamp: "Formatted at 2026-09-29 10:37:05 on dist-test-slave-vg89"
server_key: "6582a7649f35168cba1f2646a4ab55c7"
server_key_iv: "c7842b1eafe57b32b2dc4221c4a70679"
server_key_version: "encryptionkey@0"
I20260929 10:37:05.224864 327 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:37:05.240620 327 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:37:05.243918 327 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:37:05.244117 327 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:37:05.244690 327 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:37:05.245549 327 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:37:05.245615 327 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:05.245684 327 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:37:05.245730 327 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:05.254391 327 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.65:46605
I20260929 10:37:05.254472 455 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.65:46605 every 8 connection(s)
I20260929 10:37:05.255360 327 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
I20260929 10:37:05.255877 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 327
I20260929 10:37:05.256002 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/minicluster-data/ts-0/wal/instance
I20260929 10:37:05.256366 15689 external_mini_cluster.cc:1428] Setting key 4fa88d4eb51f3ca690350c6c8e817fed
Sep 29 10:37:05 dist-test-slave-vg89 krb5kdc[32688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678225, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.65@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.82.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.66@KRBTEST.COM" created.
I20260929 10:37:05.271849 304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:05.257437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.65:34553 (local address 127.15.82.126:39465)
0929 10:37:05.257594 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:05.257598 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:05.258692 (+ 1094us) server_negotiation.cc:407] Connection header received
0929 10:37:05.260759 (+ 2067us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:05.260765 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:05.260829 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:05.260888 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:05.262438 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:05.262979 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:05.264961 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:05.265194 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:05.265288 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:05.268265 (+ 2977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:05.268268 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:05.268270 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:37:05.268306 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:05.269781 (+ 1475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:05.270280 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:05.270293 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:05.270295 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:05.270376 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:05.270732 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:05.270736 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:05.270738 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:05.270970 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:05.271071 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:05.271552 (+ 481us) server_negotiation.cc:299] Negotiation successful
0929 10:37:05.271677 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260929 10:37:05.272838 458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:05.257852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:39465 (local address 127.15.82.65:34553)
0929 10:37:05.258527 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:05.258596 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0929 10:37:05.260488 (+ 1892us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:37:05.261224 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:37:05.261232 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:37:05.261587 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:37:05.262251 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:05.262274 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:05.263171 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:05.263178 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:05.264807 (+ 1629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:05.264823 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:05.265380 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:05.265384 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:05.265610 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:05.266294 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:37:05.266320 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:37:05.268097 (+ 1777us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:37:05.269905 (+ 1808us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:37:05.269910 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:37:05.269926 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:37:05.270178 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:37:05.270495 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:37:05.270498 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:37:05.270500 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:37:05.270637 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:37:05.271070 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:37:05.271077 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:37:05.271328 (+ 251us) client_negotiation.cc:769] Sending connection context
0929 10:37:05.271511 (+ 183us) client_negotiation.cc:240] Negotiation successful
0929 10:37:05.271845 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":114,"threads_started":1}
I20260929 10:37:05.274091 456 heartbeater.cc:347] Connected to a master server at 127.15.82.126:39465
I20260929 10:37:05.274346 456 heartbeater.cc:464] Registering TS with master...
I20260929 10:37:05.275032 456 heartbeater.cc:511] Master 127.15.82.126:39465 requested a full tablet report, sending...
I20260929 10:37:05.276638 32733 ts_manager.cc:194] Registered new tserver with Master: 86888832d7b541839e81ac540dcbfa74 (127.15.82.65:46605)
I20260929 10:37:05.277724 32733 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.82.65@KRBTEST.COM'} at 127.15.82.65:34553
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab.
Entry for principal kudu/127.15.82.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab.
I20260929 10:37:05.289304 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.66:0
--local_ip_for_outbound_sockets=127.15.82.66
--webserver_interface=127.15.82.66
--webserver_port=0
--tserver_master_addrs=127.15.82.126:39465
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab
--principal=kudu/127.15.82.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:38831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:37:05.399367 461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:37:05.399662 461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:37:05.399749 461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:37:05.404034 461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:37:05.404124 461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:37:05.404214 461 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.66
I20260929 10:37:05.409462 461 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:38831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab
--principal=kudu/127.15.82.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.82.66
--webserver_port=0
--tserver_master_addrs=127.15.82.126:39465
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.461
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.66
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:37:05.410591 461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:37:05.411624 461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:37:05.421207 469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:05.421198 466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:37:05.421294 461 server_base.cc:1034] running on GCE node
W20260929 10:37:05.421197 467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:37:05.421891 461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:37:05.422624 461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:37:05.423820 461 hybrid_clock.cc:648] HybridClock initialized: now 1790678225423805 us; error 29 us; skew 500 ppm
Sep 29 10:37:05 dist-test-slave-vg89 krb5kdc[32688](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678225, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:37:05.426900 461 init.cc:377] Logged in from keytab as kudu/127.15.82.66@KRBTEST.COM (short username kudu)
I20260929 10:37:05.427867 461 webserver.cc:460] Webserver started at http://127.15.82.66:42535/ using document root <none> and password file <none>
I20260929 10:37:05.428431 461 fs_manager.cc:359] Metadata directory not provided
I20260929 10:37:05.428521 461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:37:05.428762 461 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:37:05.430918 461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/minicluster-data/ts-1/data/instance:
uuid: "739d54d64025481198d1c5ee132638e8"
format_stamp: "Formatted at 2026-09-29 10:37:05 on dist-test-slave-vg89"
server_key: "7d6cd3e3f0821f247239571f59ebf74f"
server_key_iv: "655ee072f151d388737bcc8bd663d7a5"
server_key_version: "encryptionkey@0"
I20260929 10:37:05.431488 461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/minicluster-data/ts-1/wal/instance:
uuid: "739d54d64025481198d1c5ee132638e8"
format_stamp: "Formatted at 2026-09-29 10:37:05 on dist-test-slave-vg89"
server_key: "7d6cd3e3f0821f247239571f59ebf74f"
server_key_iv: "655ee072f151d388737bcc8bd663d7a5"
server_key_version: "encryptionkey@0"
I20260929 10:37:05.435163 461 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260929 10:37:05.437553 476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:05.439095 461 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 10:37:05.439220 461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/minicluster-data/ts-1/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/minicluster-data/ts-1/wal
uuid: "739d54d64025481198d1c5ee132638e8"
format_stamp: "Formatted at 2026-09-29 10:37:05 on dist-test-slave-vg89"
server_key: "7d6cd3e3f0821f247239571f59ebf74f"
server_key_iv: "655ee072f151d388737bcc8bd663d7a5"
server_key_version: "encryptionkey@0"
I20260929 10:37:05.439332 461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:37:05.476348 461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:37:05.479789 461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:37:05.479984 461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:37:05.480593 461 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:37:05.481433 461 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:37:05.481499 461 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:05.481572 461 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:37:05.481617 461 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:05.490460 461 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.66:33357
I20260929 10:37:05.490487 589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.66:33357 every 8 connection(s)
I20260929 10:37:05.491344 461 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
I20260929 10:37:05.496204 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 461
I20260929 10:37:05.496344 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/minicluster-data/ts-1/wal/instance
I20260929 10:37:05.496608 15689 external_mini_cluster.cc:1428] Setting key 5746f9c9daa8350e58137d3573c1dd65
Sep 29 10:37:05 dist-test-slave-vg89 krb5kdc[32688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678225, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.66@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:37:05.506580 304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:05.493189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:33999 (local address 127.15.82.126:39465)
0929 10:37:05.493330 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:05.493339 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:05.494701 (+ 1362us) server_negotiation.cc:407] Connection header received
0929 10:37:05.495627 (+ 926us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:05.495631 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:05.495689 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:05.495790 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:05.497046 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:05.497688 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:05.499188 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:05.499592 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:05.499712 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:05.502527 (+ 2815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:05.502534 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:05.502540 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:37:05.502575 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:05.504291 (+ 1716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:05.504815 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:05.504821 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:05.504825 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:05.504917 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:05.505278 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:05.505284 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:05.505287 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:05.505535 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:05.505636 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:05.506136 (+ 500us) server_negotiation.cc:299] Negotiation successful
0929 10:37:05.506318 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
WARNING: no policy specified for kudu/127.15.82.67@KRBTEST.COM; defaulting to no policy
I20260929 10:37:05.507369 592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:05.493627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:39465 (local address 127.15.82.66:33999)
0929 10:37:05.494554 (+ 927us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:05.494599 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0929 10:37:05.495418 (+ 819us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:37:05.495961 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:37:05.495968 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:37:05.496352 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:37:05.496902 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:05.496916 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:05.497989 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:05.497992 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:05.499058 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:05.499068 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:05.499781 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:05.499814 (+ 33us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:05.500117 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:05.500889 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:37:05.500905 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:37:05.502326 (+ 1421us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:37:05.504407 (+ 2081us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:37:05.504412 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:37:05.504424 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:37:05.504706 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:37:05.505023 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:37:05.505025 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:37:05.505027 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:37:05.505158 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:37:05.505641 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:37:05.505646 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:37:05.505883 (+ 237us) client_negotiation.cc:769] Sending connection context
0929 10:37:05.506031 (+ 148us) client_negotiation.cc:240] Negotiation successful
0929 10:37:05.506346 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":679,"thread_start_us":532,"threads_started":1}
I20260929 10:37:05.508598 590 heartbeater.cc:347] Connected to a master server at 127.15.82.126:39465
I20260929 10:37:05.508841 590 heartbeater.cc:464] Registering TS with master...
I20260929 10:37:05.509330 590 heartbeater.cc:511] Master 127.15.82.126:39465 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.67@KRBTEST.COM" created.
I20260929 10:37:05.510468 32733 ts_manager.cc:194] Registered new tserver with Master: 739d54d64025481198d1c5ee132638e8 (127.15.82.66:33357)
I20260929 10:37:05.511066 32733 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.82.66@KRBTEST.COM'} at 127.15.82.66:33999
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab.
Entry for principal kudu/127.15.82.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab.
I20260929 10:37:05.525315 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.67:0
--local_ip_for_outbound_sockets=127.15.82.67
--webserver_interface=127.15.82.67
--webserver_port=0
--tserver_master_addrs=127.15.82.126:39465
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab
--principal=kudu/127.15.82.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:38831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:37:05.642732 595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:37:05.643051 595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:37:05.643193 595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:37:05.647220 595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:37:05.647280 595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:37:05.647342 595 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.67
I20260929 10:37:05.652521 595 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:38831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab
--principal=kudu/127.15.82.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.82.67
--webserver_port=0
--tserver_master_addrs=127.15.82.126:39465
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.595
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.67
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:37:05.653532 595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:37:05.654580 595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:37:05.664768 603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:05.664768 600 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:05.664947 601 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:37:05.665247 595 server_base.cc:1034] running on GCE node
I20260929 10:37:05.665617 595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:37:05.666261 595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:37:05.667435 595 hybrid_clock.cc:648] HybridClock initialized: now 1790678225667420 us; error 33 us; skew 500 ppm
Sep 29 10:37:05 dist-test-slave-vg89 krb5kdc[32688](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678225, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:37:05.670248 595 init.cc:377] Logged in from keytab as kudu/127.15.82.67@KRBTEST.COM (short username kudu)
I20260929 10:37:05.671169 595 webserver.cc:460] Webserver started at http://127.15.82.67:34579/ using document root <none> and password file <none>
I20260929 10:37:05.671757 595 fs_manager.cc:359] Metadata directory not provided
I20260929 10:37:05.671851 595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:37:05.672088 595 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:37:05.674150 595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/minicluster-data/ts-2/data/instance:
uuid: "85653aa110064f7c9be76db206266a5e"
format_stamp: "Formatted at 2026-09-29 10:37:05 on dist-test-slave-vg89"
server_key: "c7e9bef140e6393b5b9ff7b3142711e9"
server_key_iv: "05e2abb99fc9789e8de283be6686a937"
server_key_version: "encryptionkey@0"
I20260929 10:37:05.674801 595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/minicluster-data/ts-2/wal/instance:
uuid: "85653aa110064f7c9be76db206266a5e"
format_stamp: "Formatted at 2026-09-29 10:37:05 on dist-test-slave-vg89"
server_key: "c7e9bef140e6393b5b9ff7b3142711e9"
server_key_iv: "05e2abb99fc9789e8de283be6686a937"
server_key_version: "encryptionkey@0"
I20260929 10:37:05.678589 595 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260929 10:37:05.681082 610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:05.682317 595 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 10:37:05.682474 595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/minicluster-data/ts-2/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/minicluster-data/ts-2/wal
uuid: "85653aa110064f7c9be76db206266a5e"
format_stamp: "Formatted at 2026-09-29 10:37:05 on dist-test-slave-vg89"
server_key: "c7e9bef140e6393b5b9ff7b3142711e9"
server_key_iv: "05e2abb99fc9789e8de283be6686a937"
server_key_version: "encryptionkey@0"
I20260929 10:37:05.682605 595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:37:05.710251 595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:37:05.713634 595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:37:05.713860 595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:37:05.714563 595 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:37:05.715422 595 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:37:05.715492 595 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:05.715560 595 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:37:05.715606 595 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:05.724923 595 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.67:38331
I20260929 10:37:05.724948 723 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.67:38331 every 8 connection(s)
I20260929 10:37:05.725808 595 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
I20260929 10:37:05.732402 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 595
I20260929 10:37:05.732553 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/minicluster-data/ts-2/wal/instance
I20260929 10:37:05.732874 15689 external_mini_cluster.cc:1428] Setting key edc394db6acc131171b5dd993e0d3bc3
Sep 29 10:37:05 dist-test-slave-vg89 krb5kdc[32688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678225, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.67@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:37:05.740676 304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:05.727922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.67:51349 (local address 127.15.82.126:39465)
0929 10:37:05.728085 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:05.728091 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:05.729558 (+ 1467us) server_negotiation.cc:407] Connection header received
0929 10:37:05.731061 (+ 1503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:05.731064 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:05.731124 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:05.731204 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:05.732411 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:05.733090 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:05.734577 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:05.734869 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:05.734961 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:05.737479 (+ 2518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:05.737483 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:05.737485 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:37:05.737511 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:05.738877 (+ 1366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:05.739368 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:05.739370 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:05.739371 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:05.739444 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:05.739726 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:05.739728 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:05.739729 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:05.739917 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:05.739993 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:05.740456 (+ 463us) server_negotiation.cc:299] Negotiation successful
0929 10:37:05.740556 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260929 10:37:05.741523 726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:05.728355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:39465 (local address 127.15.82.67:51349)
0929 10:37:05.729355 (+ 1000us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:05.729395 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0929 10:37:05.730840 (+ 1445us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:37:05.731397 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:37:05.731404 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:37:05.731767 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:37:05.732261 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:05.732275 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:05.733211 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:05.733215 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:05.734461 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:05.734475 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:05.735000 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:05.735005 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:05.735191 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:05.735814 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:37:05.735840 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:37:05.737329 (+ 1489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:37:05.738996 (+ 1667us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:37:05.739001 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:37:05.739014 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:37:05.739277 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:37:05.739519 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:37:05.739521 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:37:05.739522 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:37:05.739654 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:37:05.740002 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:37:05.740008 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:37:05.740240 (+ 232us) client_negotiation.cc:769] Sending connection context
0929 10:37:05.740414 (+ 174us) client_negotiation.cc:240] Negotiation successful
0929 10:37:05.740667 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":828,"thread_start_us":156,"threads_started":1}
I20260929 10:37:05.742758 724 heartbeater.cc:347] Connected to a master server at 127.15.82.126:39465
I20260929 10:37:05.743016 724 heartbeater.cc:464] Registering TS with master...
I20260929 10:37:05.743542 724 heartbeater.cc:511] Master 127.15.82.126:39465 requested a full tablet report, sending...
I20260929 10:37:05.744536 32733 ts_manager.cc:194] Registered new tserver with Master: 85653aa110064f7c9be76db206266a5e (127.15.82.67:38331)
I20260929 10:37:05.745110 32733 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.82.67@KRBTEST.COM'} at 127.15.82.67:51349
I20260929 10:37:05.746762 15689 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 10:37:05.757112 304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:05.749803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53064 (local address 127.15.82.126:39465)
0929 10:37:05.749909 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:05.749913 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:05.749931 (+ 18us) server_negotiation.cc:407] Connection header received
0929 10:37:05.749960 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:05.749962 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:05.749995 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:05.750079 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:05.750848 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:05.751358 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:05.752213 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:05.752447 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:05.752552 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:05.754074 (+ 1522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:05.754084 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:05.754087 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:37:05.754112 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:05.755413 (+ 1301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:05.755827 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:05.755833 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:05.755835 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:05.755942 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:05.756213 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:05.756217 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:05.756219 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:05.756469 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:05.756602 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:05.756833 (+ 231us) server_negotiation.cc:299] Negotiation successful
0929 10:37:05.756950 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260929 10:37:05.759994 32733 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53064:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 10:37:05.763023 32733 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:37:05.776666 735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:05.771047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.66:33357 (local address 127.0.0.1:42544)
0929 10:37:05.771470 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:05.771515 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0929 10:37:05.771644 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:37:05.772484 (+ 840us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:37:05.772491 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:37:05.772517 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:37:05.772656 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:05.772663 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:05.774011 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:05.774017 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:05.775277 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:05.775286 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:05.776080 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:05.776084 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:05.776199 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:05.776236 (+ 37us) client_negotiation.cc:769] Sending connection context
0929 10:37:05.776371 (+ 135us) client_negotiation.cc:240] Negotiation successful
0929 10:37:05.776509 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":78,"threads_started":1}
I20260929 10:37:05.776678 737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:05.771162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42544 (local address 127.15.82.66:33357)
0929 10:37:05.771571 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:05.771593 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:05.771621 (+ 28us) server_negotiation.cc:407] Connection header received
0929 10:37:05.771864 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:05.771870 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:05.772096 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:05.772275 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:37:05.772795 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:05.773888 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:05.775395 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:05.775956 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:05.776081 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:05.776249 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:05.776405 (+ 156us) server_negotiation.cc:299] Negotiation successful
0929 10:37:05.776504 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":88,"threads_started":1}
I20260929 10:37:05.777488 736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:05.771109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.65:46605 (local address 127.0.0.1:48214)
0929 10:37:05.772267 (+ 1158us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:05.772281 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 10:37:05.772387 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:37:05.773248 (+ 861us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:37:05.773251 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:37:05.773267 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:37:05.773376 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:05.773382 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:05.774723 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:05.774726 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:05.775986 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:05.775991 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:05.777066 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:05.777070 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:05.777176 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:05.777201 (+ 25us) client_negotiation.cc:769] Sending connection context
0929 10:37:05.777284 (+ 83us) client_negotiation.cc:240] Negotiation successful
0929 10:37:05.777360 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1092,"mutex_wait_us":75,"thread_start_us":112,"threads_started":1}
I20260929 10:37:05.777647 738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:05.771830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.67:38331 (local address 127.0.0.1:40634)
0929 10:37:05.772570 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:05.772583 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 10:37:05.772660 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:37:05.773054 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:37:05.773057 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:37:05.773073 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:37:05.773192 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:05.773202 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:05.774634 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:05.774641 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:05.775977 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:05.775988 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:05.777254 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:05.777258 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:05.777349 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:05.777373 (+ 24us) client_negotiation.cc:769] Sending connection context
0929 10:37:05.777451 (+ 78us) client_negotiation.cc:240] Negotiation successful
0929 10:37:05.777520 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":682,"mutex_wait_us":37,"thread_start_us":83,"threads_started":1}
I20260929 10:37:05.778155 740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:05.772214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40634 (local address 127.15.82.67:38331)
0929 10:37:05.772627 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:05.772639 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:05.772660 (+ 21us) server_negotiation.cc:407] Connection header received
0929 10:37:05.772773 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:05.772777 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:05.772929 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:05.773080 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:37:05.773287 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:05.774485 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:05.776120 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:05.776942 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:05.777049 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:05.777185 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:05.777503 (+ 318us) server_negotiation.cc:299] Negotiation successful
0929 10:37:05.778005 (+ 502us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":124,"threads_started":1}
I20260929 10:37:05.778481 739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:05.771852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48214 (local address 127.15.82.65:46605)
0929 10:37:05.772629 (+ 777us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:05.772644 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:05.772667 (+ 23us) server_negotiation.cc:407] Connection header received
0929 10:37:05.772756 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:05.772761 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:05.772943 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:05.773090 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:37:05.773496 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:05.774598 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:05.776120 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:05.776942 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:05.777049 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:05.777186 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:05.778169 (+ 983us) server_negotiation.cc:299] Negotiation successful
0929 10:37:05.778296 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":675,"thread_start_us":72,"threads_started":1}
I20260929 10:37:05.780330 524 tablet_service.cc:1467] Processing CreateTablet for tablet 1e501a2cb1c54b7c90b97bec22ea7911 (DEFAULT_TABLE table=test-table [id=b449f912ead740639eeff06362a04ebc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:37:05.780588 390 tablet_service.cc:1467] Processing CreateTablet for tablet 1e501a2cb1c54b7c90b97bec22ea7911 (DEFAULT_TABLE table=test-table [id=b449f912ead740639eeff06362a04ebc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:37:05.780963 658 tablet_service.cc:1467] Processing CreateTablet for tablet 1e501a2cb1c54b7c90b97bec22ea7911 (DEFAULT_TABLE table=test-table [id=b449f912ead740639eeff06362a04ebc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:37:05.781510 524 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e501a2cb1c54b7c90b97bec22ea7911. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:37:05.781517 390 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e501a2cb1c54b7c90b97bec22ea7911. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:37:05.781900 658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e501a2cb1c54b7c90b97bec22ea7911. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:37:05.788772 743 tablet_bootstrap.cc:492] T 1e501a2cb1c54b7c90b97bec22ea7911 P 739d54d64025481198d1c5ee132638e8: Bootstrap starting.
I20260929 10:37:05.789450 741 tablet_bootstrap.cc:492] T 1e501a2cb1c54b7c90b97bec22ea7911 P 86888832d7b541839e81ac540dcbfa74: Bootstrap starting.
I20260929 10:37:05.790746 742 tablet_bootstrap.cc:492] T 1e501a2cb1c54b7c90b97bec22ea7911 P 85653aa110064f7c9be76db206266a5e: Bootstrap starting.
I20260929 10:37:05.791164 741 tablet_bootstrap.cc:654] T 1e501a2cb1c54b7c90b97bec22ea7911 P 86888832d7b541839e81ac540dcbfa74: Neither blocks nor log segments found. Creating new log.
I20260929 10:37:05.791469 743 tablet_bootstrap.cc:654] T 1e501a2cb1c54b7c90b97bec22ea7911 P 739d54d64025481198d1c5ee132638e8: Neither blocks nor log segments found. Creating new log.
I20260929 10:37:05.791954 741 log.cc:824] T 1e501a2cb1c54b7c90b97bec22ea7911 P 86888832d7b541839e81ac540dcbfa74: Log is configured to *not* fsync() on all Append() calls
I20260929 10:37:05.792346 743 log.cc:824] T 1e501a2cb1c54b7c90b97bec22ea7911 P 739d54d64025481198d1c5ee132638e8: Log is configured to *not* fsync() on all Append() calls
I20260929 10:37:05.793022 742 tablet_bootstrap.cc:654] T 1e501a2cb1c54b7c90b97bec22ea7911 P 85653aa110064f7c9be76db206266a5e: Neither blocks nor log segments found. Creating new log.
I20260929 10:37:05.794034 741 tablet_bootstrap.cc:492] T 1e501a2cb1c54b7c90b97bec22ea7911 P 86888832d7b541839e81ac540dcbfa74: No bootstrap required, opened a new log
I20260929 10:37:05.794193 742 log.cc:824] T 1e501a2cb1c54b7c90b97bec22ea7911 P 85653aa110064f7c9be76db206266a5e: Log is configured to *not* fsync() on all Append() calls
I20260929 10:37:05.794235 741 ts_tablet_manager.cc:1397] T 1e501a2cb1c54b7c90b97bec22ea7911 P 86888832d7b541839e81ac540dcbfa74: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260929 10:37:05.794386 743 tablet_bootstrap.cc:492] T 1e501a2cb1c54b7c90b97bec22ea7911 P 739d54d64025481198d1c5ee132638e8: No bootstrap required, opened a new log
I20260929 10:37:05.794595 743 ts_tablet_manager.cc:1397] T 1e501a2cb1c54b7c90b97bec22ea7911 P 739d54d64025481198d1c5ee132638e8: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.001s
I20260929 10:37:05.795771 742 tablet_bootstrap.cc:492] T 1e501a2cb1c54b7c90b97bec22ea7911 P 85653aa110064f7c9be76db206266a5e: No bootstrap required, opened a new log
I20260929 10:37:05.795960 742 ts_tablet_manager.cc:1397] T 1e501a2cb1c54b7c90b97bec22ea7911 P 85653aa110064f7c9be76db206266a5e: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260929 10:37:05.798700 743 raft_consensus.cc:348] T 1e501a2cb1c54b7c90b97bec22ea7911 P 739d54d64025481198d1c5ee132638e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "739d54d64025481198d1c5ee132638e8" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 33357 } } peers { permanent_uuid: "86888832d7b541839e81ac540dcbfa74" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 46605 } } peers { permanent_uuid: "85653aa110064f7c9be76db206266a5e" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 38331 } }
I20260929 10:37:05.798712 741 raft_consensus.cc:348] T 1e501a2cb1c54b7c90b97bec22ea7911 P 86888832d7b541839e81ac540dcbfa74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "739d54d64025481198d1c5ee132638e8" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 33357 } } peers { permanent_uuid: "86888832d7b541839e81ac540dcbfa74" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 46605 } } peers { permanent_uuid: "85653aa110064f7c9be76db206266a5e" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 38331 } }
I20260929 10:37:05.798996 741 raft_consensus.cc:374] T 1e501a2cb1c54b7c90b97bec22ea7911 P 86888832d7b541839e81ac540dcbfa74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:37:05.798996 743 raft_consensus.cc:374] T 1e501a2cb1c54b7c90b97bec22ea7911 P 739d54d64025481198d1c5ee132638e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:37:05.799060 743 raft_consensus.cc:729] T 1e501a2cb1c54b7c90b97bec22ea7911 P 739d54d64025481198d1c5ee132638e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 739d54d64025481198d1c5ee132638e8, State: Initialized, Role: FOLLOWER
I20260929 10:37:05.799086 741 raft_consensus.cc:729] T 1e501a2cb1c54b7c90b97bec22ea7911 P 86888832d7b541839e81ac540dcbfa74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86888832d7b541839e81ac540dcbfa74, State: Initialized, Role: FOLLOWER
I20260929 10:37:05.799057 742 raft_consensus.cc:348] T 1e501a2cb1c54b7c90b97bec22ea7911 P 85653aa110064f7c9be76db206266a5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "739d54d64025481198d1c5ee132638e8" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 33357 } } peers { permanent_uuid: "86888832d7b541839e81ac540dcbfa74" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 46605 } } peers { permanent_uuid: "85653aa110064f7c9be76db206266a5e" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 38331 } }
I20260929 10:37:05.799325 742 raft_consensus.cc:374] T 1e501a2cb1c54b7c90b97bec22ea7911 P 85653aa110064f7c9be76db206266a5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:37:05.799386 742 raft_consensus.cc:729] T 1e501a2cb1c54b7c90b97bec22ea7911 P 85653aa110064f7c9be76db206266a5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85653aa110064f7c9be76db206266a5e, State: Initialized, Role: FOLLOWER
I20260929 10:37:05.799608 741 consensus_queue.cc:260] T 1e501a2cb1c54b7c90b97bec22ea7911 P 86888832d7b541839e81ac540dcbfa74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "739d54d64025481198d1c5ee132638e8" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 33357 } } peers { permanent_uuid: "86888832d7b541839e81ac540dcbfa74" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 46605 } } peers { permanent_uuid: "85653aa110064f7c9be76db206266a5e" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 38331 } }
I20260929 10:37:05.799603 743 consensus_queue.cc:260] T 1e501a2cb1c54b7c90b97bec22ea7911 P 739d54d64025481198d1c5ee132638e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "739d54d64025481198d1c5ee132638e8" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 33357 } } peers { permanent_uuid: "86888832d7b541839e81ac540dcbfa74" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 46605 } } peers { permanent_uuid: "85653aa110064f7c9be76db206266a5e" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 38331 } }
I20260929 10:37:05.799876 742 consensus_queue.cc:260] T 1e501a2cb1c54b7c90b97bec22ea7911 P 85653aa110064f7c9be76db206266a5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "739d54d64025481198d1c5ee132638e8" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 33357 } } peers { permanent_uuid: "86888832d7b541839e81ac540dcbfa74" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 46605 } } peers { permanent_uuid: "85653aa110064f7c9be76db206266a5e" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 38331 } }
I20260929 10:37:05.800611 743 ts_tablet_manager.cc:1428] T 1e501a2cb1c54b7c90b97bec22ea7911 P 739d54d64025481198d1c5ee132638e8: Time spent starting tablet: real 0.006s user 0.003s sys 0.002s
I20260929 10:37:05.800611 741 ts_tablet_manager.cc:1428] T 1e501a2cb1c54b7c90b97bec22ea7911 P 86888832d7b541839e81ac540dcbfa74: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260929 10:37:05.800866 590 heartbeater.cc:503] Master 127.15.82.126:39465 was elected leader, sending a full tablet report...
I20260929 10:37:05.800904 742 ts_tablet_manager.cc:1428] T 1e501a2cb1c54b7c90b97bec22ea7911 P 85653aa110064f7c9be76db206266a5e: Time spent starting tablet: real 0.005s user 0.001s sys 0.005s
I20260929 10:37:05.800936 456 heartbeater.cc:503] Master 127.15.82.126:39465 was elected leader, sending a full tablet report...
I20260929 10:37:05.801198 724 heartbeater.cc:503] Master 127.15.82.126:39465 was elected leader, sending a full tablet report...
I20260929 10:37:05.853073 747 raft_consensus.cc:482] T 1e501a2cb1c54b7c90b97bec22ea7911 P 739d54d64025481198d1c5ee132638e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:37:05.853274 747 raft_consensus.cc:504] T 1e501a2cb1c54b7c90b97bec22ea7911 P 739d54d64025481198d1c5ee132638e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "739d54d64025481198d1c5ee132638e8" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 33357 } } peers { permanent_uuid: "86888832d7b541839e81ac540dcbfa74" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 46605 } } peers { permanent_uuid: "85653aa110064f7c9be76db206266a5e" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 38331 } }
I20260929 10:37:05.854374 747 leader_election.cc:288] T 1e501a2cb1c54b7c90b97bec22ea7911 P 739d54d64025481198d1c5ee132638e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 86888832d7b541839e81ac540dcbfa74 (127.15.82.65:46605), 85653aa110064f7c9be76db206266a5e (127.15.82.67:38331)
I20260929 10:37:05.858093 740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:05.854722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:53547 (local address 127.15.82.67:38331)
0929 10:37:05.854846 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:05.854852 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:05.854868 (+ 16us) server_negotiation.cc:407] Connection header received
0929 10:37:05.855042 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:05.855045 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:05.855085 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:05.855164 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:37:05.855433 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:05.855992 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:05.857133 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:05.857544 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:05.857671 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:05.857728 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:05.857906 (+ 178us) server_negotiation.cc:299] Negotiation successful
0929 10:37:05.857975 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260929 10:37:05.858104 592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:05.854654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.67:38331 (local address 127.15.82.66:53547)
0929 10:37:05.854768 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:05.854820 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0929 10:37:05.854912 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:37:05.855176 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:37:05.855179 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:37:05.855195 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:37:05.855322 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:05.855327 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:05.856095 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:05.856101 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:05.857002 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:05.857011 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:05.857663 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:05.857667 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:05.857770 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:05.857799 (+ 29us) client_negotiation.cc:769] Sending connection context
0929 10:37:05.857889 (+ 90us) client_negotiation.cc:240] Negotiation successful
0929 10:37:05.857958 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":1,"spinlock_wait_cycles":1920}
I20260929 10:37:05.858525 750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:05.854657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.65:46605 (local address 127.15.82.66:59913)
0929 10:37:05.855096 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:05.855107 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:37:05.855218 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:37:05.855482 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:37:05.855485 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:37:05.855499 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:37:05.855650 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:05.855655 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:05.856435 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:05.856441 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:05.857373 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:05.857382 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:05.858047 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:05.858051 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:05.858146 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:05.858173 (+ 27us) client_negotiation.cc:769] Sending connection context
0929 10:37:05.858301 (+ 128us) client_negotiation.cc:240] Negotiation successful
0929 10:37:05.858363 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":379,"mutex_wait_us":117,"thread_start_us":163,"threads_started":1}
I20260929 10:37:05.858541 739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:05.854763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:59913 (local address 127.15.82.65:46605)
0929 10:37:05.854893 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:05.854899 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:05.855184 (+ 285us) server_negotiation.cc:407] Connection header received
0929 10:37:05.855329 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:05.855334 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:05.855382 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:05.855460 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:37:05.855760 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:05.856311 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:05.857493 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:05.857931 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:05.858052 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:05.858100 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:05.858323 (+ 223us) server_negotiation.cc:299] Negotiation successful
0929 10:37:05.858424 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260929 10:37:05.858836 678 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e501a2cb1c54b7c90b97bec22ea7911" candidate_uuid: "739d54d64025481198d1c5ee132638e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85653aa110064f7c9be76db206266a5e" is_pre_election: true
I20260929 10:37:05.859189 678 raft_consensus.cc:2454] T 1e501a2cb1c54b7c90b97bec22ea7911 P 85653aa110064f7c9be76db206266a5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 739d54d64025481198d1c5ee132638e8 in term 0.
I20260929 10:37:05.859179 410 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e501a2cb1c54b7c90b97bec22ea7911" candidate_uuid: "739d54d64025481198d1c5ee132638e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86888832d7b541839e81ac540dcbfa74" is_pre_election: true
I20260929 10:37:05.859464 410 raft_consensus.cc:2454] T 1e501a2cb1c54b7c90b97bec22ea7911 P 86888832d7b541839e81ac540dcbfa74 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 739d54d64025481198d1c5ee132638e8 in term 0.
I20260929 10:37:05.859730 479 leader_election.cc:302] T 1e501a2cb1c54b7c90b97bec22ea7911 P 739d54d64025481198d1c5ee132638e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 739d54d64025481198d1c5ee132638e8, 85653aa110064f7c9be76db206266a5e; no voters:
I20260929 10:37:05.860033 747 raft_consensus.cc:2781] T 1e501a2cb1c54b7c90b97bec22ea7911 P 739d54d64025481198d1c5ee132638e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:37:05.860124 747 raft_consensus.cc:482] T 1e501a2cb1c54b7c90b97bec22ea7911 P 739d54d64025481198d1c5ee132638e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:37:05.860172 747 raft_consensus.cc:3037] T 1e501a2cb1c54b7c90b97bec22ea7911 P 739d54d64025481198d1c5ee132638e8 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:37:05.861168 747 raft_consensus.cc:504] T 1e501a2cb1c54b7c90b97bec22ea7911 P 739d54d64025481198d1c5ee132638e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "739d54d64025481198d1c5ee132638e8" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 33357 } } peers { permanent_uuid: "86888832d7b541839e81ac540dcbfa74" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 46605 } } peers { permanent_uuid: "85653aa110064f7c9be76db206266a5e" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 38331 } }
I20260929 10:37:05.861485 747 leader_election.cc:288] T 1e501a2cb1c54b7c90b97bec22ea7911 P 739d54d64025481198d1c5ee132638e8 [CANDIDATE]: Term 1 election: Requested vote from peers 86888832d7b541839e81ac540dcbfa74 (127.15.82.65:46605), 85653aa110064f7c9be76db206266a5e (127.15.82.67:38331)
I20260929 10:37:05.861814 410 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e501a2cb1c54b7c90b97bec22ea7911" candidate_uuid: "739d54d64025481198d1c5ee132638e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86888832d7b541839e81ac540dcbfa74"
I20260929 10:37:05.861814 678 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e501a2cb1c54b7c90b97bec22ea7911" candidate_uuid: "739d54d64025481198d1c5ee132638e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85653aa110064f7c9be76db206266a5e"
I20260929 10:37:05.861937 678 raft_consensus.cc:3037] T 1e501a2cb1c54b7c90b97bec22ea7911 P 85653aa110064f7c9be76db206266a5e [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:37:05.861938 410 raft_consensus.cc:3037] T 1e501a2cb1c54b7c90b97bec22ea7911 P 86888832d7b541839e81ac540dcbfa74 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:37:05.863255 410 raft_consensus.cc:2454] T 1e501a2cb1c54b7c90b97bec22ea7911 P 86888832d7b541839e81ac540dcbfa74 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 739d54d64025481198d1c5ee132638e8 in term 1.
I20260929 10:37:05.863263 678 raft_consensus.cc:2454] T 1e501a2cb1c54b7c90b97bec22ea7911 P 85653aa110064f7c9be76db206266a5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 739d54d64025481198d1c5ee132638e8 in term 1.
I20260929 10:37:05.863554 479 leader_election.cc:302] T 1e501a2cb1c54b7c90b97bec22ea7911 P 739d54d64025481198d1c5ee132638e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 739d54d64025481198d1c5ee132638e8, 85653aa110064f7c9be76db206266a5e; no voters:
I20260929 10:37:05.863731 747 raft_consensus.cc:2781] T 1e501a2cb1c54b7c90b97bec22ea7911 P 739d54d64025481198d1c5ee132638e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:37:05.863963 747 raft_consensus.cc:686] T 1e501a2cb1c54b7c90b97bec22ea7911 P 739d54d64025481198d1c5ee132638e8 [term 1 LEADER]: Becoming Leader. State: Replica: 739d54d64025481198d1c5ee132638e8, State: Running, Role: LEADER
I20260929 10:37:05.864286 747 consensus_queue.cc:237] T 1e501a2cb1c54b7c90b97bec22ea7911 P 739d54d64025481198d1c5ee132638e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "739d54d64025481198d1c5ee132638e8" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 33357 } } peers { permanent_uuid: "86888832d7b541839e81ac540dcbfa74" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 46605 } } peers { permanent_uuid: "85653aa110064f7c9be76db206266a5e" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 38331 } }
I20260929 10:37:05.867442 32733 catalog_manager.cc:5495] T 1e501a2cb1c54b7c90b97bec22ea7911 P 739d54d64025481198d1c5ee132638e8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 739d54d64025481198d1c5ee132638e8 (127.15.82.66). New cstate: current_term: 1 leader_uuid: "739d54d64025481198d1c5ee132638e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "739d54d64025481198d1c5ee132638e8" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 33357 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "86888832d7b541839e81ac540dcbfa74" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 46605 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85653aa110064f7c9be76db206266a5e" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 38331 } health_report { overall_health: UNKNOWN } } }
I20260929 10:37:05.900847 737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:05.897175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42558 (local address 127.15.82.66:33357)
0929 10:37:05.897304 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:05.897308 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:05.897321 (+ 13us) server_negotiation.cc:407] Connection header received
0929 10:37:05.897398 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:05.897400 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:05.897455 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:05.897547 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:37:05.897796 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:05.898468 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:05.899402 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:05.899662 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:05.899757 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:05.900083 (+ 326us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:37:05.900522 (+ 439us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:37:05.900615 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:05.900678 (+ 63us) server_negotiation.cc:299] Negotiation successful
0929 10:37:05.900731 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260929 10:37:05.906708 678 raft_consensus.cc:1261] T 1e501a2cb1c54b7c90b97bec22ea7911 P 85653aa110064f7c9be76db206266a5e [term 1 FOLLOWER]: Refusing update from remote peer 739d54d64025481198d1c5ee132638e8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:37:05.906708 410 raft_consensus.cc:1261] T 1e501a2cb1c54b7c90b97bec22ea7911 P 86888832d7b541839e81ac540dcbfa74 [term 1 FOLLOWER]: Refusing update from remote peer 739d54d64025481198d1c5ee132638e8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:37:05.907397 751 consensus_queue.cc:1035] T 1e501a2cb1c54b7c90b97bec22ea7911 P 739d54d64025481198d1c5ee132638e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "86888832d7b541839e81ac540dcbfa74" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 46605 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:37:05.907588 747 consensus_queue.cc:1035] T 1e501a2cb1c54b7c90b97bec22ea7911 P 739d54d64025481198d1c5ee132638e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85653aa110064f7c9be76db206266a5e" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 38331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:37:05.915060 756 mvcc.cc:205] Tried to move back new op lower bound from 7334618013306826752 to 7334618013142822912. Current Snapshot: MvccSnapshot[applied={T|T < 7334618013306826752}]
I20260929 10:37:05.922595 32731 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53064:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:37:05.922777 32731 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53064:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:37:05.925242 32731 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0794cecd6ef9491892ee1c23f39d317a: Sending DeleteTablet for 3 replicas of tablet 1e501a2cb1c54b7c90b97bec22ea7911
I20260929 10:37:05.925974 390 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e501a2cb1c54b7c90b97bec22ea7911 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:37:05 UTC) from {username='kudu', principal='kudu/127.15.82.126@KRBTEST.COM'} at 127.0.0.1:48214
I20260929 10:37:05.926016 524 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e501a2cb1c54b7c90b97bec22ea7911 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:37:05 UTC) from {username='kudu', principal='kudu/127.15.82.126@KRBTEST.COM'} at 127.0.0.1:42544
I20260929 10:37:05.926079 658 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e501a2cb1c54b7c90b97bec22ea7911 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:37:05 UTC) from {username='kudu', principal='kudu/127.15.82.126@KRBTEST.COM'} at 127.0.0.1:40634
I20260929 10:37:05.926648 763 tablet_replica.cc:330] T 1e501a2cb1c54b7c90b97bec22ea7911 P 86888832d7b541839e81ac540dcbfa74: stopping tablet replica
I20260929 10:37:05.926960 763 raft_consensus.cc:2229] T 1e501a2cb1c54b7c90b97bec22ea7911 P 86888832d7b541839e81ac540dcbfa74 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:37:05.927168 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 327
I20260929 10:37:05.927300 763 raft_consensus.cc:2258] T 1e501a2cb1c54b7c90b97bec22ea7911 P 86888832d7b541839e81ac540dcbfa74 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:37:05.927762 765 tablet_replica.cc:330] T 1e501a2cb1c54b7c90b97bec22ea7911 P 739d54d64025481198d1c5ee132638e8: stopping tablet replica
I20260929 10:37:05.927975 765 raft_consensus.cc:2229] T 1e501a2cb1c54b7c90b97bec22ea7911 P 739d54d64025481198d1c5ee132638e8 [term 1 LEADER]: Raft consensus shutting down.
I20260929 10:37:05.928239 765 raft_consensus.cc:2258] T 1e501a2cb1c54b7c90b97bec22ea7911 P 739d54d64025481198d1c5ee132638e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:37:05.928536 764 tablet_replica.cc:330] T 1e501a2cb1c54b7c90b97bec22ea7911 P 85653aa110064f7c9be76db206266a5e: stopping tablet replica
I20260929 10:37:05.928805 764 raft_consensus.cc:2229] T 1e501a2cb1c54b7c90b97bec22ea7911 P 85653aa110064f7c9be76db206266a5e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:37:05.929059 764 raft_consensus.cc:2258] T 1e501a2cb1c54b7c90b97bec22ea7911 P 85653aa110064f7c9be76db206266a5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:37:05.929620 765 ts_tablet_manager.cc:1905] T 1e501a2cb1c54b7c90b97bec22ea7911 P 739d54d64025481198d1c5ee132638e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:37:05.933004 765 ts_tablet_manager.cc:1918] T 1e501a2cb1c54b7c90b97bec22ea7911 P 739d54d64025481198d1c5ee132638e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:37:05.933143 765 log.cc:1196] T 1e501a2cb1c54b7c90b97bec22ea7911 P 739d54d64025481198d1c5ee132638e8: Deleting WAL directory at /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/minicluster-data/ts-1/wal/wals/1e501a2cb1c54b7c90b97bec22ea7911
I20260929 10:37:05.933506 765 ts_tablet_manager.cc:1939] T 1e501a2cb1c54b7c90b97bec22ea7911 P 739d54d64025481198d1c5ee132638e8: Deleting consensus metadata
I20260929 10:37:05.935262 32720 catalog_manager.cc:4841] TS 739d54d64025481198d1c5ee132638e8 (127.15.82.66:33357): tablet 1e501a2cb1c54b7c90b97bec22ea7911 (table test-table [id=b449f912ead740639eeff06362a04ebc]) successfully deleted
W20260929 10:37:05.936065 32718 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.82.65:46605 (error 108)
I20260929 10:37:05.936316 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 461
I20260929 10:37:05.939196 738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:05.938924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.65:46605 (local address 127.0.0.1:48226)
0929 10:37:05.939052 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:05.939125 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.65:46605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260929 10:37:05.939191 764 ts_tablet_manager.cc:1905] T 1e501a2cb1c54b7c90b97bec22ea7911 P 85653aa110064f7c9be76db206266a5e: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260929 10:37:05.939441 32718 catalog_manager.cc:4584] TS 86888832d7b541839e81ac540dcbfa74 (127.15.82.65:46605): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1e501a2cb1c54b7c90b97bec22ea7911: Network error: Client connection negotiation failed: client connection to 127.15.82.65:46605: connect: Connection refused (error 111)
I20260929 10:37:05.942634 764 ts_tablet_manager.cc:1918] T 1e501a2cb1c54b7c90b97bec22ea7911 P 85653aa110064f7c9be76db206266a5e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:37:05.942744 764 log.cc:1196] T 1e501a2cb1c54b7c90b97bec22ea7911 P 85653aa110064f7c9be76db206266a5e: Deleting WAL directory at /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790677981592896-15689-0/minicluster-data/ts-2/wal/wals/1e501a2cb1c54b7c90b97bec22ea7911
I20260929 10:37:05.943110 764 ts_tablet_manager.cc:1939] T 1e501a2cb1c54b7c90b97bec22ea7911 P 85653aa110064f7c9be76db206266a5e: Deleting consensus metadata
I20260929 10:37:05.944090 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 595
I20260929 10:37:05.949754 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 32701
2026-09-29T10:37:05Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2310 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-29T10:37:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:37:05Z Disabled control of system clock
I20260929 10:37:05.980551 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790677981592896-15689-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790677981592896-15689-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790677981592896-15689-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:39649
--webserver_interface=127.15.82.126
--webserver_port=0
--builtin_ntp_servers=127.15.82.84:35703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.82.126:39649
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 10:37:06.089567 770 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:37:06.089851 770 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:37:06.089886 770 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:37:06.093806 770 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:37:06.093864 770 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:37:06.093880 770 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:37:06.093896 770 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:37:06.093912 770 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:37:06.098803 770 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:35703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790677981592896-15689-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790677981592896-15689-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.82.126:39649
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:39649
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.82.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.770
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790677981592896-15689-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:37:06.099871 770 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:37:06.101236 770 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:37:06.109655 778 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:06.109663 775 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:06.109656 776 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:37:06.109961 770 server_base.cc:1034] running on GCE node
I20260929 10:37:06.110637 770 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:37:06.111754 770 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:37:06.112915 770 hybrid_clock.cc:648] HybridClock initialized: now 1790678226112902 us; error 25 us; skew 500 ppm
I20260929 10:37:06.114768 770 webserver.cc:460] Webserver started at http://127.15.82.126:42521/ using document root <none> and password file <none>
I20260929 10:37:06.115379 770 fs_manager.cc:359] Metadata directory not provided
I20260929 10:37:06.115486 770 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:37:06.115721 770 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:37:06.117827 770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790677981592896-15689-0/minicluster-data/master-0/data/instance:
uuid: "73866b54b7e14ecc88bd5a00749fe0b2"
format_stamp: "Formatted at 2026-09-29 10:37:06 on dist-test-slave-vg89"
server_key: "29e69ddd9b52077d691c013ef73bff5c"
server_key_iv: "2386c1a965df0af2b40beb212ed25dee"
server_key_version: "encryptionkey@0"
I20260929 10:37:06.118440 770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790677981592896-15689-0/minicluster-data/master-0/wal/instance:
uuid: "73866b54b7e14ecc88bd5a00749fe0b2"
format_stamp: "Formatted at 2026-09-29 10:37:06 on dist-test-slave-vg89"
server_key: "29e69ddd9b52077d691c013ef73bff5c"
server_key_iv: "2386c1a965df0af2b40beb212ed25dee"
server_key_version: "encryptionkey@0"
I20260929 10:37:06.122071 770 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260929 10:37:06.124567 784 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:06.125797 770 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 10:37:06.125924 770 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790677981592896-15689-0/minicluster-data/master-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790677981592896-15689-0/minicluster-data/master-0/wal
uuid: "73866b54b7e14ecc88bd5a00749fe0b2"
format_stamp: "Formatted at 2026-09-29 10:37:06 on dist-test-slave-vg89"
server_key: "29e69ddd9b52077d691c013ef73bff5c"
server_key_iv: "2386c1a965df0af2b40beb212ed25dee"
server_key_version: "encryptionkey@0"
I20260929 10:37:06.126034 770 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790677981592896-15689-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790677981592896-15689-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790677981592896-15689-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:37:06.137583 770 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:37:06.138239 770 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:37:06.138470 770 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:37:06.145119 770 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.126:39649
I20260929 10:37:06.145126 836 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.126:39649 every 8 connection(s)
I20260929 10:37:06.146085 770 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
I20260929 10:37:06.149197 837 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:37:06.154743 837 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73866b54b7e14ecc88bd5a00749fe0b2: Bootstrap starting.
I20260929 10:37:06.156193 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 770
I20260929 10:37:06.156328 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790677981592896-15689-0/minicluster-data/master-0/wal/instance
I20260929 10:37:06.156610 15689 external_mini_cluster.cc:1428] Setting key 03ccb7f7b1782d5743362b14dd11d576
I20260929 10:37:06.157339 837 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 73866b54b7e14ecc88bd5a00749fe0b2: Neither blocks nor log segments found. Creating new log.
I20260929 10:37:06.158308 837 log.cc:824] T 00000000000000000000000000000000 P 73866b54b7e14ecc88bd5a00749fe0b2: Log is configured to *not* fsync() on all Append() calls
I20260929 10:37:06.160555 837 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73866b54b7e14ecc88bd5a00749fe0b2: No bootstrap required, opened a new log
I20260929 10:37:06.164855 837 raft_consensus.cc:348] T 00000000000000000000000000000000 P 73866b54b7e14ecc88bd5a00749fe0b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73866b54b7e14ecc88bd5a00749fe0b2" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 39649 } }
I20260929 10:37:06.165146 837 raft_consensus.cc:374] T 00000000000000000000000000000000 P 73866b54b7e14ecc88bd5a00749fe0b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:37:06.165215 837 raft_consensus.cc:729] T 00000000000000000000000000000000 P 73866b54b7e14ecc88bd5a00749fe0b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73866b54b7e14ecc88bd5a00749fe0b2, State: Initialized, Role: FOLLOWER
I20260929 10:37:06.165709 837 consensus_queue.cc:260] T 00000000000000000000000000000000 P 73866b54b7e14ecc88bd5a00749fe0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73866b54b7e14ecc88bd5a00749fe0b2" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 39649 } }
I20260929 10:37:06.165869 837 raft_consensus.cc:388] T 00000000000000000000000000000000 P 73866b54b7e14ecc88bd5a00749fe0b2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:37:06.165959 837 raft_consensus.cc:482] T 00000000000000000000000000000000 P 73866b54b7e14ecc88bd5a00749fe0b2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:37:06.166065 837 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 73866b54b7e14ecc88bd5a00749fe0b2 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:37:06.166726 840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:06.157710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36078 (local address 127.15.82.126:39649)
0929 10:37:06.158380 (+ 670us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:06.158439 (+ 59us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:06.158494 (+ 55us) server_negotiation.cc:407] Connection header received
0929 10:37:06.159276 (+ 782us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:06.159299 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:06.159720 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:06.160079 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:06.160517 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:06.161987 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:06.162969 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:06.163705 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:06.163913 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:06.164640 (+ 727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:06.164646 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:06.164661 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:37:06.165147 (+ 486us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:06.165222 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:37:06.165252 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:06.165401 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:06.165551 (+ 150us) server_negotiation.cc:299] Negotiation successful
0929 10:37:06.165815 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":244,"thread_start_us":113,"threads_started":1}
I20260929 10:37:06.167073 837 raft_consensus.cc:504] T 00000000000000000000000000000000 P 73866b54b7e14ecc88bd5a00749fe0b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73866b54b7e14ecc88bd5a00749fe0b2" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 39649 } }
I20260929 10:37:06.167397 837 leader_election.cc:302] T 00000000000000000000000000000000 P 73866b54b7e14ecc88bd5a00749fe0b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 73866b54b7e14ecc88bd5a00749fe0b2; no voters:
I20260929 10:37:06.167707 837 leader_election.cc:288] T 00000000000000000000000000000000 P 73866b54b7e14ecc88bd5a00749fe0b2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:37:06.167773 842 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 73866b54b7e14ecc88bd5a00749fe0b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:37:06.167968 842 raft_consensus.cc:686] T 00000000000000000000000000000000 P 73866b54b7e14ecc88bd5a00749fe0b2 [term 1 LEADER]: Becoming Leader. State: Replica: 73866b54b7e14ecc88bd5a00749fe0b2, State: Running, Role: LEADER
I20260929 10:37:06.168313 842 consensus_queue.cc:237] T 00000000000000000000000000000000 P 73866b54b7e14ecc88bd5a00749fe0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73866b54b7e14ecc88bd5a00749fe0b2" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 39649 } }
I20260929 10:37:06.168978 837 sys_catalog.cc:564] T 00000000000000000000000000000000 P 73866b54b7e14ecc88bd5a00749fe0b2 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:37:06.169533 844 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73866b54b7e14ecc88bd5a00749fe0b2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "73866b54b7e14ecc88bd5a00749fe0b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73866b54b7e14ecc88bd5a00749fe0b2" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 39649 } } }
I20260929 10:37:06.169663 844 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73866b54b7e14ecc88bd5a00749fe0b2 [sys.catalog]: This master's current role is: LEADER
I20260929 10:37:06.169994 843 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73866b54b7e14ecc88bd5a00749fe0b2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 73866b54b7e14ecc88bd5a00749fe0b2. Latest consensus state: current_term: 1 leader_uuid: "73866b54b7e14ecc88bd5a00749fe0b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73866b54b7e14ecc88bd5a00749fe0b2" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 39649 } } }
I20260929 10:37:06.170109 843 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73866b54b7e14ecc88bd5a00749fe0b2 [sys.catalog]: This master's current role is: LEADER
I20260929 10:37:06.170691 847 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:37:06.173277 847 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:37:06.179306 847 catalog_manager.cc:1347] Generated new cluster ID: 023315ade58444608d66fb31eb2cc5b8
I20260929 10:37:06.179368 847 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:37:06.189633 847 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:37:06.190861 847 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:37:06.204859 847 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 73866b54b7e14ecc88bd5a00749fe0b2: Generated new TSK 0
I20260929 10:37:06.205691 847 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 10:37:06.209743 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790677981592896-15689-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790677981592896-15689-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790677981592896-15689-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.65:0
--local_ip_for_outbound_sockets=127.15.82.65
--webserver_interface=127.15.82.65
--webserver_port=0
--tserver_master_addrs=127.15.82.126:39649
--builtin_ntp_servers=127.15.82.84:35703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 10:37:06.319722 861 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:37:06.319977 861 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:37:06.320016 861 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:37:06.323985 861 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:37:06.324033 861 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:37:06.324084 861 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.65
I20260929 10:37:06.328917 861 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:35703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790677981592896-15689-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790677981592896-15689-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.82.65
--webserver_port=0
--tserver_master_addrs=127.15.82.126:39649
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.861
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.65
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790677981592896-15689-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:37:06.330086 861 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:37:06.331564 861 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:37:06.340986 867 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:06.341054 869 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:06.340999 866 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:37:06.341250 861 server_base.cc:1034] running on GCE node
I20260929 10:37:06.341683 861 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:37:06.342308 861 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:37:06.343465 861 hybrid_clock.cc:648] HybridClock initialized: now 1790678226343448 us; error 28 us; skew 500 ppm
I20260929 10:37:06.345227 861 webserver.cc:460] Webserver started at http://127.15.82.65:46317/ using document root <none> and password file <none>
I20260929 10:37:06.345796 861 fs_manager.cc:359] Metadata directory not provided
I20260929 10:37:06.345881 861 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:37:06.346087 861 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:37:06.348135 861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790677981592896-15689-0/minicluster-data/ts-0/data/instance:
uuid: "3203cbbffdea4401b85aea8372493be4"
format_stamp: "Formatted at 2026-09-29 10:37:06 on dist-test-slave-vg89"
server_key: "4c48073eeeba6c23b100517d37bc372c"
server_key_iv: "d84dfb866ceb178f8216ab21e270a4a6"
server_key_version: "encryptionkey@0"
I20260929 10:37:06.348681 861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790677981592896-15689-0/minicluster-data/ts-0/wal/instance:
uuid: "3203cbbffdea4401b85aea8372493be4"
format_stamp: "Formatted at 2026-09-29 10:37:06 on dist-test-slave-vg89"
server_key: "4c48073eeeba6c23b100517d37bc372c"
server_key_iv: "d84dfb866ceb178f8216ab21e270a4a6"
server_key_version: "encryptionkey@0"
I20260929 10:37:06.352439 861 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260929 10:37:06.354962 875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:06.356137 861 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 10:37:06.356261 861 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790677981592896-15689-0/minicluster-data/ts-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790677981592896-15689-0/minicluster-data/ts-0/wal
uuid: "3203cbbffdea4401b85aea8372493be4"
format_stamp: "Formatted at 2026-09-29 10:37:06 on dist-test-slave-vg89"
server_key: "4c48073eeeba6c23b100517d37bc372c"
server_key_iv: "d84dfb866ceb178f8216ab21e270a4a6"
server_key_version: "encryptionkey@0"
I20260929 10:37:06.356369 861 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790677981592896-15689-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790677981592896-15689-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790677981592896-15689-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:37:06.366504 861 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:37:06.367095 861 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:37:06.367272 861 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:37:06.367810 861 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:37:06.368796 861 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:37:06.368870 861 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:06.368937 861 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:37:06.368984 861 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:06.377779 861 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.65:34453
I20260929 10:37:06.377815 988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.65:34453 every 8 connection(s)
I20260929 10:37:06.378713 861 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
I20260929 10:37:06.386428 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 861
I20260929 10:37:06.386557 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790677981592896-15689-0/minicluster-data/ts-0/wal/instance
I20260929 10:37:06.386860 15689 external_mini_cluster.cc:1428] Setting key 66622d14c49046099b2a7b571d961d06
I20260929 10:37:06.388031 840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:06.380502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.65:51559 (local address 127.15.82.126:39649)
0929 10:37:06.380643 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:06.380647 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:06.381662 (+ 1015us) server_negotiation.cc:407] Connection header received
0929 10:37:06.383129 (+ 1467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:06.383132 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:06.383192 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:06.383300 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:06.384177 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:06.384735 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:06.385867 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:06.386098 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:06.386211 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:06.387498 (+ 1287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:06.387502 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:06.387506 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:37:06.387524 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:06.387547 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:37:06.387557 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:06.387623 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:06.387831 (+ 208us) server_negotiation.cc:299] Negotiation successful
0929 10:37:06.387892 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 10:37:06.388989 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790677981592896-15689-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790677981592896-15689-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790677981592896-15689-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.66:0
--local_ip_for_outbound_sockets=127.15.82.66
--webserver_interface=127.15.82.66
--webserver_port=0
--tserver_master_addrs=127.15.82.126:39649
--builtin_ntp_servers=127.15.82.84:35703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260929 10:37:06.389068 991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:06.380893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:39649 (local address 127.15.82.65:51559)
0929 10:37:06.381516 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:06.381556 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0929 10:37:06.382912 (+ 1356us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:37:06.383465 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:37:06.383472 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:37:06.383555 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:37:06.384033 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:06.384046 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:06.384861 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:06.384865 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:06.385734 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:06.385742 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:06.386260 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:06.386265 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:06.386499 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:06.387178 (+ 679us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 10:37:06.387181 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:37:06.387235 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 10:37:06.387315 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:37:06.387652 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:37:06.387661 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:37:06.387669 (+ 8us) client_negotiation.cc:769] Sending connection context
0929 10:37:06.387799 (+ 130us) client_negotiation.cc:240] Negotiation successful
0929 10:37:06.388111 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":107,"threads_started":1}
I20260929 10:37:06.390388 989 heartbeater.cc:347] Connected to a master server at 127.15.82.126:39649
I20260929 10:37:06.390691 989 heartbeater.cc:464] Registering TS with master...
I20260929 10:37:06.391222 989 heartbeater.cc:511] Master 127.15.82.126:39649 requested a full tablet report, sending...
I20260929 10:37:06.392699 801 ts_manager.cc:194] Registered new tserver with Master: 3203cbbffdea4401b85aea8372493be4 (127.15.82.65:34453)
I20260929 10:37:06.393875 801 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.82.65:51559
W20260929 10:37:06.501619 992 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:37:06.501925 992 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:37:06.501994 992 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:37:06.506083 992 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:37:06.506176 992 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:37:06.506263 992 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.66
I20260929 10:37:06.511072 992 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:35703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790677981592896-15689-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790677981592896-15689-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.82.66
--webserver_port=0
--tserver_master_addrs=127.15.82.126:39649
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.992
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.66
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790677981592896-15689-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:37:06.512163 992 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:37:06.513564 992 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:37:06.523950 998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:06.523973 1000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:06.523948 997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:37:06.524022 992 server_base.cc:1034] running on GCE node
I20260929 10:37:06.524547 992 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:37:06.525153 992 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:37:06.526378 992 hybrid_clock.cc:648] HybridClock initialized: now 1790678226526360 us; error 32 us; skew 500 ppm
I20260929 10:37:06.528367 992 webserver.cc:460] Webserver started at http://127.15.82.66:44429/ using document root <none> and password file <none>
I20260929 10:37:06.529114 992 fs_manager.cc:359] Metadata directory not provided
I20260929 10:37:06.529202 992 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:37:06.529461 992 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:37:06.531680 992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790677981592896-15689-0/minicluster-data/ts-1/data/instance:
uuid: "5d64a778ffa44370931238b948061bad"
format_stamp: "Formatted at 2026-09-29 10:37:06 on dist-test-slave-vg89"
server_key: "c43ece1418edeaab28166927f14f827f"
server_key_iv: "444fb18bcd19e4a1a03335711ff0a1bd"
server_key_version: "encryptionkey@0"
I20260929 10:37:06.532248 992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790677981592896-15689-0/minicluster-data/ts-1/wal/instance:
uuid: "5d64a778ffa44370931238b948061bad"
format_stamp: "Formatted at 2026-09-29 10:37:06 on dist-test-slave-vg89"
server_key: "c43ece1418edeaab28166927f14f827f"
server_key_iv: "444fb18bcd19e4a1a03335711ff0a1bd"
server_key_version: "encryptionkey@0"
I20260929 10:37:06.536019 992 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 10:37:06.538378 1006 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:06.539645 992 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 10:37:06.539780 992 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790677981592896-15689-0/minicluster-data/ts-1/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790677981592896-15689-0/minicluster-data/ts-1/wal
uuid: "5d64a778ffa44370931238b948061bad"
format_stamp: "Formatted at 2026-09-29 10:37:06 on dist-test-slave-vg89"
server_key: "c43ece1418edeaab28166927f14f827f"
server_key_iv: "444fb18bcd19e4a1a03335711ff0a1bd"
server_key_version: "encryptionkey@0"
I20260929 10:37:06.539892 992 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790677981592896-15689-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790677981592896-15689-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790677981592896-15689-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:37:06.549881 992 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:37:06.550603 992 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:37:06.550789 992 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:37:06.551350 992 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:37:06.552278 992 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:37:06.552347 992 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:06.552407 992 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:37:06.552452 992 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:06.561131 992 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.66:37137
I20260929 10:37:06.561182 1119 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.66:37137 every 8 connection(s)
I20260929 10:37:06.562027 992 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
I20260929 10:37:06.566006 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 992
I20260929 10:37:06.566161 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790677981592896-15689-0/minicluster-data/ts-1/wal/instance
I20260929 10:37:06.566473 15689 external_mini_cluster.cc:1428] Setting key ee14e43e32c7c081023c430ddb65a855
I20260929 10:37:06.568382 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790677981592896-15689-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790677981592896-15689-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790677981592896-15689-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.67:0
--local_ip_for_outbound_sockets=127.15.82.67
--webserver_interface=127.15.82.67
--webserver_port=0
--tserver_master_addrs=127.15.82.126:39649
--builtin_ntp_servers=127.15.82.84:35703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260929 10:37:06.571810 840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:06.563914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:34077 (local address 127.15.82.126:39649)
0929 10:37:06.564049 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:06.564053 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:06.565112 (+ 1059us) server_negotiation.cc:407] Connection header received
0929 10:37:06.566667 (+ 1555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:06.566670 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:06.566740 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:06.566835 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:06.567721 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:06.568244 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:06.569552 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:06.569836 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:06.569941 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:06.571123 (+ 1182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:06.571129 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:06.571135 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:37:06.571151 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:06.571178 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:37:06.571191 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:06.571281 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:06.571529 (+ 248us) server_negotiation.cc:299] Negotiation successful
0929 10:37:06.571603 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260929 10:37:06.572579 1122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:06.564290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:39649 (local address 127.15.82.66:34077)
0929 10:37:06.564952 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:06.564993 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 10:37:06.566446 (+ 1453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:37:06.567007 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:37:06.567013 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:37:06.567097 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:37:06.567585 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:06.567597 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:06.568375 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:06.568378 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:06.569422 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:06.569432 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:06.569978 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:06.569983 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:06.570153 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:06.570854 (+ 701us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 10:37:06.570856 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:37:06.570895 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 10:37:06.570980 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:37:06.571335 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:37:06.571360 (+ 25us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:37:06.571367 (+ 7us) client_negotiation.cc:769] Sending connection context
0929 10:37:06.571517 (+ 150us) client_negotiation.cc:240] Negotiation successful
0929 10:37:06.571747 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":107,"threads_started":1}
I20260929 10:37:06.573655 1120 heartbeater.cc:347] Connected to a master server at 127.15.82.126:39649
I20260929 10:37:06.573912 1120 heartbeater.cc:464] Registering TS with master...
I20260929 10:37:06.574615 1120 heartbeater.cc:511] Master 127.15.82.126:39649 requested a full tablet report, sending...
I20260929 10:37:06.575767 801 ts_manager.cc:194] Registered new tserver with Master: 5d64a778ffa44370931238b948061bad (127.15.82.66:37137)
I20260929 10:37:06.576365 801 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.82.66:34077
W20260929 10:37:06.680768 1123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:37:06.681016 1123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:37:06.681053 1123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:37:06.685151 1123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:37:06.685204 1123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:37:06.685261 1123 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.67
I20260929 10:37:06.689980 1123 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:35703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790677981592896-15689-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790677981592896-15689-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.82.67
--webserver_port=0
--tserver_master_addrs=127.15.82.126:39649
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1123
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.67
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790677981592896-15689-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:37:06.691004 1123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:37:06.692320 1123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:37:06.701756 1131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:06.701812 1128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:06.701735 1129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:37:06.701941 1123 server_base.cc:1034] running on GCE node
I20260929 10:37:06.702368 1123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:37:06.703091 1123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:37:06.704309 1123 hybrid_clock.cc:648] HybridClock initialized: now 1790678226704271 us; error 51 us; skew 500 ppm
I20260929 10:37:06.706401 1123 webserver.cc:460] Webserver started at http://127.15.82.67:40905/ using document root <none> and password file <none>
I20260929 10:37:06.707098 1123 fs_manager.cc:359] Metadata directory not provided
I20260929 10:37:06.707170 1123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:37:06.707356 1123 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:37:06.709522 1123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790677981592896-15689-0/minicluster-data/ts-2/data/instance:
uuid: "8a054ab9a11c412bad3e1afa16a950ce"
format_stamp: "Formatted at 2026-09-29 10:37:06 on dist-test-slave-vg89"
server_key: "281914414a1cd6be410b9d9309815fd4"
server_key_iv: "d0cd89bf5e9f93519c06ebe5ae663f25"
server_key_version: "encryptionkey@0"
I20260929 10:37:06.710062 1123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790677981592896-15689-0/minicluster-data/ts-2/wal/instance:
uuid: "8a054ab9a11c412bad3e1afa16a950ce"
format_stamp: "Formatted at 2026-09-29 10:37:06 on dist-test-slave-vg89"
server_key: "281914414a1cd6be410b9d9309815fd4"
server_key_iv: "d0cd89bf5e9f93519c06ebe5ae663f25"
server_key_version: "encryptionkey@0"
I20260929 10:37:06.713959 1123 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260929 10:37:06.716771 1137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:06.718073 1123 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 10:37:06.718216 1123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790677981592896-15689-0/minicluster-data/ts-2/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790677981592896-15689-0/minicluster-data/ts-2/wal
uuid: "8a054ab9a11c412bad3e1afa16a950ce"
format_stamp: "Formatted at 2026-09-29 10:37:06 on dist-test-slave-vg89"
server_key: "281914414a1cd6be410b9d9309815fd4"
server_key_iv: "d0cd89bf5e9f93519c06ebe5ae663f25"
server_key_version: "encryptionkey@0"
I20260929 10:37:06.718338 1123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790677981592896-15689-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790677981592896-15689-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790677981592896-15689-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:37:06.744267 1123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:37:06.744992 1123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:37:06.745148 1123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:37:06.745704 1123 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:37:06.746888 1123 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:37:06.746946 1123 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:06.747018 1123 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:37:06.747054 1123 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:06.756289 1123 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.67:43371
I20260929 10:37:06.756317 1250 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.67:43371 every 8 connection(s)
I20260929 10:37:06.757194 1123 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
I20260929 10:37:06.766031 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 1123
I20260929 10:37:06.766146 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790677981592896-15689-0/minicluster-data/ts-2/wal/instance
I20260929 10:37:06.766187 840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:06.758994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.67:35935 (local address 127.15.82.126:39649)
0929 10:37:06.759125 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:06.759129 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:06.760281 (+ 1152us) server_negotiation.cc:407] Connection header received
0929 10:37:06.761716 (+ 1435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:06.761719 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:06.761775 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:06.761859 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:06.762794 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:06.763317 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:06.764385 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:06.764594 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:06.764668 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:06.765633 (+ 965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:06.765636 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:06.765639 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:37:06.765651 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:06.765670 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:37:06.765677 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:06.765751 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:06.765986 (+ 235us) server_negotiation.cc:299] Negotiation successful
0929 10:37:06.766036 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260929 10:37:06.766494 15689 external_mini_cluster.cc:1428] Setting key 02333e6b6036fc946b21b7b923ab75fe
I20260929 10:37:06.767273 1253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:06.759397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:39649 (local address 127.15.82.67:35935)
0929 10:37:06.760129 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:06.760180 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0929 10:37:06.761529 (+ 1349us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:37:06.762014 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:37:06.762021 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:37:06.762103 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:37:06.762668 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:06.762683 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:06.763421 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:06.763423 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:06.764278 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:06.764286 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:06.764716 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:06.764718 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:06.764852 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:06.765404 (+ 552us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 10:37:06.765406 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:37:06.765443 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 10:37:06.765501 (+ 58us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:37:06.765804 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:37:06.765818 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:37:06.765826 (+ 8us) client_negotiation.cc:769] Sending connection context
0929 10:37:06.765993 (+ 167us) client_negotiation.cc:240] Negotiation successful
0929 10:37:06.766321 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":161,"threads_started":1}
I20260929 10:37:06.768389 1251 heartbeater.cc:347] Connected to a master server at 127.15.82.126:39649
I20260929 10:37:06.768635 1251 heartbeater.cc:464] Registering TS with master...
I20260929 10:37:06.769129 1251 heartbeater.cc:511] Master 127.15.82.126:39649 requested a full tablet report, sending...
I20260929 10:37:06.770218 796 ts_manager.cc:194] Registered new tserver with Master: 8a054ab9a11c412bad3e1afa16a950ce (127.15.82.67:43371)
I20260929 10:37:06.770890 796 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.82.67:35935
I20260929 10:37:06.780544 15689 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 10:37:06.786404 840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:06.782532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36086 (local address 127.15.82.126:39649)
0929 10:37:06.782670 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:06.782674 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:06.782853 (+ 179us) server_negotiation.cc:407] Connection header received
0929 10:37:06.782992 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:06.782995 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:06.783039 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:06.783112 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:06.783343 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:06.783844 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:06.784684 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:06.784943 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:06.785023 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:06.785881 (+ 858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:06.785884 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:06.785886 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:37:06.785898 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:06.785917 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:37:06.785925 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:06.785993 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:06.786146 (+ 153us) server_negotiation.cc:299] Negotiation successful
0929 10:37:06.786209 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260929 10:37:06.789409 796 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36086:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 10:37:06.792639 796 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:37:06.807000 1265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:06.801990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.66:37137 (local address 127.0.0.1:38856)
0929 10:37:06.802615 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:06.802629 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 10:37:06.802722 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:37:06.803204 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:37:06.803207 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:37:06.803224 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:37:06.803326 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:06.803336 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:06.804278 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:06.804284 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:06.805502 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:06.805511 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:06.806445 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:06.806449 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:06.806550 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:06.806591 (+ 41us) client_negotiation.cc:769] Sending connection context
0929 10:37:06.806704 (+ 113us) client_negotiation.cc:240] Negotiation successful
0929 10:37:06.806832 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":87,"threads_started":1}
I20260929 10:37:06.807032 1267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:06.802387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38856 (local address 127.15.82.66:37137)
0929 10:37:06.802846 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:06.802856 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:06.802874 (+ 18us) server_negotiation.cc:407] Connection header received
0929 10:37:06.802940 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:06.802943 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:06.803080 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:06.803231 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:37:06.803458 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:06.804170 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:06.805626 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:06.806301 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:06.806430 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:06.806619 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:06.806729 (+ 110us) server_negotiation.cc:299] Negotiation successful
0929 10:37:06.806832 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":102,"threads_started":1}
I20260929 10:37:06.807482 1264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:06.801697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.67:43371 (local address 127.0.0.1:56424)
0929 10:37:06.802495 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:06.802510 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 10:37:06.802594 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:37:06.803333 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:37:06.803338 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:37:06.803357 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:37:06.803449 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:06.803454 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:06.804887 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:06.804891 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:06.806048 (+ 1157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:06.806062 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:06.807039 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:06.807044 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:06.807191 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:06.807216 (+ 25us) client_negotiation.cc:769] Sending connection context
0929 10:37:06.807263 (+ 47us) client_negotiation.cc:240] Negotiation successful
0929 10:37:06.807333 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"mutex_wait_us":6,"thread_start_us":106,"threads_started":1}
I20260929 10:37:06.807670 1262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:06.801229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.65:34453 (local address 127.0.0.1:60914)
0929 10:37:06.801596 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:06.801649 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0929 10:37:06.801789 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:37:06.802812 (+ 1023us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:37:06.802819 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:37:06.802842 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:37:06.802986 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:06.802992 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:06.804604 (+ 1612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:06.804607 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:06.805917 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:06.805927 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:06.806971 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:06.806974 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:06.807347 (+ 373us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:06.807366 (+ 19us) client_negotiation.cc:769] Sending connection context
0929 10:37:06.807461 (+ 95us) client_negotiation.cc:240] Negotiation successful
0929 10:37:06.807535 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"mutex_wait_us":346,"thread_start_us":87,"threads_started":1}
I20260929 10:37:06.808061 1263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:06.801426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60914 (local address 127.15.82.65:34453)
0929 10:37:06.802240 (+ 814us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:06.802257 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:06.802299 (+ 42us) server_negotiation.cc:407] Connection header received
0929 10:37:06.802381 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:06.802387 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:06.802636 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:06.803135 (+ 499us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:37:06.803300 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:06.804475 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:06.806049 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:06.806831 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:06.807098 (+ 267us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:06.807289 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:06.807504 (+ 215us) server_negotiation.cc:299] Negotiation successful
0929 10:37:06.807601 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":707,"thread_start_us":134,"threads_started":1}
I20260929 10:37:06.808686 1266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:06.802121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56424 (local address 127.15.82.67:43371)
0929 10:37:06.802874 (+ 753us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:06.802886 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:06.802908 (+ 22us) server_negotiation.cc:407] Connection header received
0929 10:37:06.802978 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:06.802983 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:06.803181 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:06.803587 (+ 406us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:37:06.803713 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:06.804758 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:06.806166 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:06.806904 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:06.807747 (+ 843us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:06.808435 (+ 688us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:06.808492 (+ 57us) server_negotiation.cc:299] Negotiation successful
0929 10:37:06.808569 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":637,"thread_start_us":151,"threads_started":1}
I20260929 10:37:06.810840 1185 tablet_service.cc:1467] Processing CreateTablet for tablet c8f16f881d034a7bbafc7f08e1d80599 (DEFAULT_TABLE table=test-table [id=912e9883dbe74af2a93dbb0ddb4faf9c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:37:06.810675 1054 tablet_service.cc:1467] Processing CreateTablet for tablet c8f16f881d034a7bbafc7f08e1d80599 (DEFAULT_TABLE table=test-table [id=912e9883dbe74af2a93dbb0ddb4faf9c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:37:06.810675 923 tablet_service.cc:1467] Processing CreateTablet for tablet c8f16f881d034a7bbafc7f08e1d80599 (DEFAULT_TABLE table=test-table [id=912e9883dbe74af2a93dbb0ddb4faf9c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:37:06.811784 923 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8f16f881d034a7bbafc7f08e1d80599. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:37:06.811784 1054 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8f16f881d034a7bbafc7f08e1d80599. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:37:06.811791 1185 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8f16f881d034a7bbafc7f08e1d80599. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:37:06.817873 1268 tablet_bootstrap.cc:492] T c8f16f881d034a7bbafc7f08e1d80599 P 8a054ab9a11c412bad3e1afa16a950ce: Bootstrap starting.
I20260929 10:37:06.819447 1269 tablet_bootstrap.cc:492] T c8f16f881d034a7bbafc7f08e1d80599 P 3203cbbffdea4401b85aea8372493be4: Bootstrap starting.
I20260929 10:37:06.819447 1270 tablet_bootstrap.cc:492] T c8f16f881d034a7bbafc7f08e1d80599 P 5d64a778ffa44370931238b948061bad: Bootstrap starting.
I20260929 10:37:06.820098 1268 tablet_bootstrap.cc:654] T c8f16f881d034a7bbafc7f08e1d80599 P 8a054ab9a11c412bad3e1afa16a950ce: Neither blocks nor log segments found. Creating new log.
I20260929 10:37:06.820822 1268 log.cc:824] T c8f16f881d034a7bbafc7f08e1d80599 P 8a054ab9a11c412bad3e1afa16a950ce: Log is configured to *not* fsync() on all Append() calls
I20260929 10:37:06.821538 1270 tablet_bootstrap.cc:654] T c8f16f881d034a7bbafc7f08e1d80599 P 5d64a778ffa44370931238b948061bad: Neither blocks nor log segments found. Creating new log.
I20260929 10:37:06.821539 1269 tablet_bootstrap.cc:654] T c8f16f881d034a7bbafc7f08e1d80599 P 3203cbbffdea4401b85aea8372493be4: Neither blocks nor log segments found. Creating new log.
I20260929 10:37:06.822445 1269 log.cc:824] T c8f16f881d034a7bbafc7f08e1d80599 P 3203cbbffdea4401b85aea8372493be4: Log is configured to *not* fsync() on all Append() calls
I20260929 10:37:06.822438 1270 log.cc:824] T c8f16f881d034a7bbafc7f08e1d80599 P 5d64a778ffa44370931238b948061bad: Log is configured to *not* fsync() on all Append() calls
I20260929 10:37:06.823072 1268 tablet_bootstrap.cc:492] T c8f16f881d034a7bbafc7f08e1d80599 P 8a054ab9a11c412bad3e1afa16a950ce: No bootstrap required, opened a new log
I20260929 10:37:06.823339 1268 ts_tablet_manager.cc:1397] T c8f16f881d034a7bbafc7f08e1d80599 P 8a054ab9a11c412bad3e1afa16a950ce: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260929 10:37:06.824373 1269 tablet_bootstrap.cc:492] T c8f16f881d034a7bbafc7f08e1d80599 P 3203cbbffdea4401b85aea8372493be4: No bootstrap required, opened a new log
I20260929 10:37:06.824373 1270 tablet_bootstrap.cc:492] T c8f16f881d034a7bbafc7f08e1d80599 P 5d64a778ffa44370931238b948061bad: No bootstrap required, opened a new log
I20260929 10:37:06.824564 1269 ts_tablet_manager.cc:1397] T c8f16f881d034a7bbafc7f08e1d80599 P 3203cbbffdea4401b85aea8372493be4: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260929 10:37:06.824564 1270 ts_tablet_manager.cc:1397] T c8f16f881d034a7bbafc7f08e1d80599 P 5d64a778ffa44370931238b948061bad: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260929 10:37:06.826678 1268 raft_consensus.cc:348] T c8f16f881d034a7bbafc7f08e1d80599 P 8a054ab9a11c412bad3e1afa16a950ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3203cbbffdea4401b85aea8372493be4" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34453 } } peers { permanent_uuid: "5d64a778ffa44370931238b948061bad" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 37137 } } peers { permanent_uuid: "8a054ab9a11c412bad3e1afa16a950ce" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 43371 } }
I20260929 10:37:06.826918 1268 raft_consensus.cc:374] T c8f16f881d034a7bbafc7f08e1d80599 P 8a054ab9a11c412bad3e1afa16a950ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:37:06.826988 1268 raft_consensus.cc:729] T c8f16f881d034a7bbafc7f08e1d80599 P 8a054ab9a11c412bad3e1afa16a950ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a054ab9a11c412bad3e1afa16a950ce, State: Initialized, Role: FOLLOWER
I20260929 10:37:06.827457 1268 consensus_queue.cc:260] T c8f16f881d034a7bbafc7f08e1d80599 P 8a054ab9a11c412bad3e1afa16a950ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3203cbbffdea4401b85aea8372493be4" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34453 } } peers { permanent_uuid: "5d64a778ffa44370931238b948061bad" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 37137 } } peers { permanent_uuid: "8a054ab9a11c412bad3e1afa16a950ce" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 43371 } }
I20260929 10:37:06.828114 1251 heartbeater.cc:503] Master 127.15.82.126:39649 was elected leader, sending a full tablet report...
I20260929 10:37:06.828421 1268 ts_tablet_manager.cc:1428] T c8f16f881d034a7bbafc7f08e1d80599 P 8a054ab9a11c412bad3e1afa16a950ce: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260929 10:37:06.828773 1269 raft_consensus.cc:348] T c8f16f881d034a7bbafc7f08e1d80599 P 3203cbbffdea4401b85aea8372493be4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3203cbbffdea4401b85aea8372493be4" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34453 } } peers { permanent_uuid: "5d64a778ffa44370931238b948061bad" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 37137 } } peers { permanent_uuid: "8a054ab9a11c412bad3e1afa16a950ce" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 43371 } }
I20260929 10:37:06.829102 1269 raft_consensus.cc:374] T c8f16f881d034a7bbafc7f08e1d80599 P 3203cbbffdea4401b85aea8372493be4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:37:06.829195 1269 raft_consensus.cc:729] T c8f16f881d034a7bbafc7f08e1d80599 P 3203cbbffdea4401b85aea8372493be4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3203cbbffdea4401b85aea8372493be4, State: Initialized, Role: FOLLOWER
I20260929 10:37:06.829346 1270 raft_consensus.cc:348] T c8f16f881d034a7bbafc7f08e1d80599 P 5d64a778ffa44370931238b948061bad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3203cbbffdea4401b85aea8372493be4" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34453 } } peers { permanent_uuid: "5d64a778ffa44370931238b948061bad" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 37137 } } peers { permanent_uuid: "8a054ab9a11c412bad3e1afa16a950ce" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 43371 } }
I20260929 10:37:06.829712 1270 raft_consensus.cc:374] T c8f16f881d034a7bbafc7f08e1d80599 P 5d64a778ffa44370931238b948061bad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:37:06.829658 1269 consensus_queue.cc:260] T c8f16f881d034a7bbafc7f08e1d80599 P 3203cbbffdea4401b85aea8372493be4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3203cbbffdea4401b85aea8372493be4" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34453 } } peers { permanent_uuid: "5d64a778ffa44370931238b948061bad" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 37137 } } peers { permanent_uuid: "8a054ab9a11c412bad3e1afa16a950ce" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 43371 } }
I20260929 10:37:06.829777 1270 raft_consensus.cc:729] T c8f16f881d034a7bbafc7f08e1d80599 P 5d64a778ffa44370931238b948061bad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d64a778ffa44370931238b948061bad, State: Initialized, Role: FOLLOWER
I20260929 10:37:06.830190 1270 consensus_queue.cc:260] T c8f16f881d034a7bbafc7f08e1d80599 P 5d64a778ffa44370931238b948061bad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3203cbbffdea4401b85aea8372493be4" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34453 } } peers { permanent_uuid: "5d64a778ffa44370931238b948061bad" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 37137 } } peers { permanent_uuid: "8a054ab9a11c412bad3e1afa16a950ce" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 43371 } }
I20260929 10:37:06.830377 989 heartbeater.cc:503] Master 127.15.82.126:39649 was elected leader, sending a full tablet report...
I20260929 10:37:06.830678 1269 ts_tablet_manager.cc:1428] T c8f16f881d034a7bbafc7f08e1d80599 P 3203cbbffdea4401b85aea8372493be4: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260929 10:37:06.831188 1270 ts_tablet_manager.cc:1428] T c8f16f881d034a7bbafc7f08e1d80599 P 5d64a778ffa44370931238b948061bad: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260929 10:37:06.831543 1120 heartbeater.cc:503] Master 127.15.82.126:39649 was elected leader, sending a full tablet report...
W20260929 10:37:06.879884 990 tablet.cc:2369] T c8f16f881d034a7bbafc7f08e1d80599 P 3203cbbffdea4401b85aea8372493be4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 10:37:06.990005 1274 raft_consensus.cc:482] T c8f16f881d034a7bbafc7f08e1d80599 P 8a054ab9a11c412bad3e1afa16a950ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:37:06.990239 1274 raft_consensus.cc:504] T c8f16f881d034a7bbafc7f08e1d80599 P 8a054ab9a11c412bad3e1afa16a950ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3203cbbffdea4401b85aea8372493be4" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34453 } } peers { permanent_uuid: "5d64a778ffa44370931238b948061bad" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 37137 } } peers { permanent_uuid: "8a054ab9a11c412bad3e1afa16a950ce" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 43371 } }
I20260929 10:37:06.991397 1274 leader_election.cc:288] T c8f16f881d034a7bbafc7f08e1d80599 P 8a054ab9a11c412bad3e1afa16a950ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3203cbbffdea4401b85aea8372493be4 (127.15.82.65:34453), 5d64a778ffa44370931238b948061bad (127.15.82.66:37137)
I20260929 10:37:06.996079 1253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:06.991722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.65:34453 (local address 127.15.82.67:57563)
0929 10:37:06.991876 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:06.991893 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 10:37:06.992007 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:37:06.992327 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:37:06.992331 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:37:06.992356 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:37:06.992508 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:06.992518 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:06.993626 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:06.993629 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:06.994864 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:06.994882 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:06.995708 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:06.995711 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:06.995782 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:06.995802 (+ 20us) client_negotiation.cc:769] Sending connection context
0929 10:37:06.995859 (+ 57us) client_negotiation.cc:240] Negotiation successful
0929 10:37:06.995923 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260929 10:37:06.996121 1263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:06.991887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.67:57563 (local address 127.15.82.65:34453)
0929 10:37:06.992025 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:06.992029 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:06.992077 (+ 48us) server_negotiation.cc:407] Connection header received
0929 10:37:06.992130 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:06.992132 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:06.992196 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:06.992296 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:37:06.992649 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:06.993489 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:06.995018 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:06.995589 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:06.995715 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:06.995766 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:06.995899 (+ 133us) server_negotiation.cc:299] Negotiation successful
0929 10:37:06.995989 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260929 10:37:06.996450 1267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:06.992310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.67:50665 (local address 127.15.82.66:37137)
0929 10:37:06.992465 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:06.992468 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:06.992615 (+ 147us) server_negotiation.cc:407] Connection header received
0929 10:37:06.992780 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:06.992784 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:06.992837 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:06.992929 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:37:06.993169 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:06.993868 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:06.995221 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:06.995899 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:06.996013 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:06.996056 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:06.996271 (+ 215us) server_negotiation.cc:299] Negotiation successful
0929 10:37:06.996356 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260929 10:37:06.996798 1277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:06.992049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.66:37137 (local address 127.15.82.67:50665)
0929 10:37:06.992511 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:06.992526 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 10:37:06.992628 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:37:06.992943 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:37:06.992945 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:37:06.992958 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:37:06.993060 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:06.993065 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:06.993984 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:06.993987 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:06.995105 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:06.995112 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:06.996018 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:06.996022 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:06.996113 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:06.996140 (+ 27us) client_negotiation.cc:769] Sending connection context
0929 10:37:06.996601 (+ 461us) client_negotiation.cc:240] Negotiation successful
0929 10:37:06.996674 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":388,"mutex_wait_us":3,"thread_start_us":131,"threads_started":1}
I20260929 10:37:06.996827 943 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8f16f881d034a7bbafc7f08e1d80599" candidate_uuid: "8a054ab9a11c412bad3e1afa16a950ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3203cbbffdea4401b85aea8372493be4" is_pre_election: true
I20260929 10:37:06.997155 943 raft_consensus.cc:2454] T c8f16f881d034a7bbafc7f08e1d80599 P 3203cbbffdea4401b85aea8372493be4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a054ab9a11c412bad3e1afa16a950ce in term 0.
I20260929 10:37:06.997418 1074 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8f16f881d034a7bbafc7f08e1d80599" candidate_uuid: "8a054ab9a11c412bad3e1afa16a950ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d64a778ffa44370931238b948061bad" is_pre_election: true
I20260929 10:37:06.997706 1074 raft_consensus.cc:2454] T c8f16f881d034a7bbafc7f08e1d80599 P 5d64a778ffa44370931238b948061bad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a054ab9a11c412bad3e1afa16a950ce in term 0.
I20260929 10:37:06.997759 1138 leader_election.cc:302] T c8f16f881d034a7bbafc7f08e1d80599 P 8a054ab9a11c412bad3e1afa16a950ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3203cbbffdea4401b85aea8372493be4, 8a054ab9a11c412bad3e1afa16a950ce; no voters:
I20260929 10:37:06.998095 1274 raft_consensus.cc:2781] T c8f16f881d034a7bbafc7f08e1d80599 P 8a054ab9a11c412bad3e1afa16a950ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:37:06.998198 1274 raft_consensus.cc:482] T c8f16f881d034a7bbafc7f08e1d80599 P 8a054ab9a11c412bad3e1afa16a950ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:37:06.998258 1274 raft_consensus.cc:3037] T c8f16f881d034a7bbafc7f08e1d80599 P 8a054ab9a11c412bad3e1afa16a950ce [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:37:06.999300 1274 raft_consensus.cc:504] T c8f16f881d034a7bbafc7f08e1d80599 P 8a054ab9a11c412bad3e1afa16a950ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3203cbbffdea4401b85aea8372493be4" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34453 } } peers { permanent_uuid: "5d64a778ffa44370931238b948061bad" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 37137 } } peers { permanent_uuid: "8a054ab9a11c412bad3e1afa16a950ce" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 43371 } }
I20260929 10:37:06.999624 1274 leader_election.cc:288] T c8f16f881d034a7bbafc7f08e1d80599 P 8a054ab9a11c412bad3e1afa16a950ce [CANDIDATE]: Term 1 election: Requested vote from peers 3203cbbffdea4401b85aea8372493be4 (127.15.82.65:34453), 5d64a778ffa44370931238b948061bad (127.15.82.66:37137)
I20260929 10:37:06.999951 943 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8f16f881d034a7bbafc7f08e1d80599" candidate_uuid: "8a054ab9a11c412bad3e1afa16a950ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3203cbbffdea4401b85aea8372493be4"
I20260929 10:37:07.000028 1074 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8f16f881d034a7bbafc7f08e1d80599" candidate_uuid: "8a054ab9a11c412bad3e1afa16a950ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d64a778ffa44370931238b948061bad"
I20260929 10:37:07.000082 943 raft_consensus.cc:3037] T c8f16f881d034a7bbafc7f08e1d80599 P 3203cbbffdea4401b85aea8372493be4 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:37:07.000168 1074 raft_consensus.cc:3037] T c8f16f881d034a7bbafc7f08e1d80599 P 5d64a778ffa44370931238b948061bad [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:37:07.001132 1074 raft_consensus.cc:2454] T c8f16f881d034a7bbafc7f08e1d80599 P 5d64a778ffa44370931238b948061bad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a054ab9a11c412bad3e1afa16a950ce in term 1.
I20260929 10:37:07.001135 943 raft_consensus.cc:2454] T c8f16f881d034a7bbafc7f08e1d80599 P 3203cbbffdea4401b85aea8372493be4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a054ab9a11c412bad3e1afa16a950ce in term 1.
I20260929 10:37:07.001448 1138 leader_election.cc:302] T c8f16f881d034a7bbafc7f08e1d80599 P 8a054ab9a11c412bad3e1afa16a950ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3203cbbffdea4401b85aea8372493be4, 8a054ab9a11c412bad3e1afa16a950ce; no voters:
I20260929 10:37:07.001631 1274 raft_consensus.cc:2781] T c8f16f881d034a7bbafc7f08e1d80599 P 8a054ab9a11c412bad3e1afa16a950ce [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:37:07.001865 1274 raft_consensus.cc:686] T c8f16f881d034a7bbafc7f08e1d80599 P 8a054ab9a11c412bad3e1afa16a950ce [term 1 LEADER]: Becoming Leader. State: Replica: 8a054ab9a11c412bad3e1afa16a950ce, State: Running, Role: LEADER
I20260929 10:37:07.002209 1274 consensus_queue.cc:237] T c8f16f881d034a7bbafc7f08e1d80599 P 8a054ab9a11c412bad3e1afa16a950ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3203cbbffdea4401b85aea8372493be4" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34453 } } peers { permanent_uuid: "5d64a778ffa44370931238b948061bad" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 37137 } } peers { permanent_uuid: "8a054ab9a11c412bad3e1afa16a950ce" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 43371 } }
I20260929 10:37:07.005460 801 catalog_manager.cc:5495] T c8f16f881d034a7bbafc7f08e1d80599 P 8a054ab9a11c412bad3e1afa16a950ce reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a054ab9a11c412bad3e1afa16a950ce (127.15.82.67). New cstate: current_term: 1 leader_uuid: "8a054ab9a11c412bad3e1afa16a950ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3203cbbffdea4401b85aea8372493be4" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34453 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d64a778ffa44370931238b948061bad" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 37137 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a054ab9a11c412bad3e1afa16a950ce" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 43371 } health_report { overall_health: HEALTHY } } }
W20260929 10:37:07.008215 1252 tablet.cc:2369] T c8f16f881d034a7bbafc7f08e1d80599 P 8a054ab9a11c412bad3e1afa16a950ce: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 10:37:07.041075 1266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:07.037521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56428 (local address 127.15.82.67:43371)
0929 10:37:07.037658 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:07.037664 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:07.037679 (+ 15us) server_negotiation.cc:407] Connection header received
0929 10:37:07.037778 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:07.037781 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:07.037844 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:07.037959 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:37:07.038187 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:07.038784 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:07.039619 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:07.039847 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:07.039952 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:07.040249 (+ 297us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:37:07.040671 (+ 422us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:37:07.040802 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:07.040861 (+ 59us) server_negotiation.cc:299] Negotiation successful
0929 10:37:07.040915 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260929 10:37:07.047187 943 raft_consensus.cc:1261] T c8f16f881d034a7bbafc7f08e1d80599 P 3203cbbffdea4401b85aea8372493be4 [term 1 FOLLOWER]: Refusing update from remote peer 8a054ab9a11c412bad3e1afa16a950ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:37:07.047195 1074 raft_consensus.cc:1261] T c8f16f881d034a7bbafc7f08e1d80599 P 5d64a778ffa44370931238b948061bad [term 1 FOLLOWER]: Refusing update from remote peer 8a054ab9a11c412bad3e1afa16a950ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:37:07.047902 1274 consensus_queue.cc:1035] T c8f16f881d034a7bbafc7f08e1d80599 P 8a054ab9a11c412bad3e1afa16a950ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "3203cbbffdea4401b85aea8372493be4" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 34453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:37:07.048107 1278 consensus_queue.cc:1035] T c8f16f881d034a7bbafc7f08e1d80599 P 8a054ab9a11c412bad3e1afa16a950ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d64a778ffa44370931238b948061bad" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 37137 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:37:07.056504 1282 mvcc.cc:205] Tried to move back new op lower bound from 7334618017977004032 to 7334618017804075008. Current Snapshot: MvccSnapshot[applied={T|T < 7334618017977004032}]
I20260929 10:37:07.057047 1283 mvcc.cc:205] Tried to move back new op lower bound from 7334618017977004032 to 7334618017804075008. Current Snapshot: MvccSnapshot[applied={T|T < 7334618017977004032}]
I20260929 10:37:07.065472 796 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:36086:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:37:07.065653 796 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:36086:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:37:07.068321 796 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 73866b54b7e14ecc88bd5a00749fe0b2: Sending DeleteTablet for 3 replicas of tablet c8f16f881d034a7bbafc7f08e1d80599
I20260929 10:37:07.069141 923 tablet_service.cc:1514] Processing DeleteTablet for tablet c8f16f881d034a7bbafc7f08e1d80599 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:37:07 UTC) from {username='slave'} at 127.0.0.1:60914
I20260929 10:37:07.069222 1185 tablet_service.cc:1514] Processing DeleteTablet for tablet c8f16f881d034a7bbafc7f08e1d80599 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:37:07 UTC) from {username='slave'} at 127.0.0.1:56424
I20260929 10:37:07.069208 1054 tablet_service.cc:1514] Processing DeleteTablet for tablet c8f16f881d034a7bbafc7f08e1d80599 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:37:07 UTC) from {username='slave'} at 127.0.0.1:38856
I20260929 10:37:07.069710 1291 tablet_replica.cc:330] T c8f16f881d034a7bbafc7f08e1d80599 P 3203cbbffdea4401b85aea8372493be4: stopping tablet replica
I20260929 10:37:07.069977 1293 tablet_replica.cc:330] T c8f16f881d034a7bbafc7f08e1d80599 P 8a054ab9a11c412bad3e1afa16a950ce: stopping tablet replica
I20260929 10:37:07.070104 1291 raft_consensus.cc:2229] T c8f16f881d034a7bbafc7f08e1d80599 P 3203cbbffdea4401b85aea8372493be4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:37:07.070302 1293 raft_consensus.cc:2229] T c8f16f881d034a7bbafc7f08e1d80599 P 8a054ab9a11c412bad3e1afa16a950ce [term 1 LEADER]: Raft consensus shutting down.
I20260929 10:37:07.070482 1291 raft_consensus.cc:2258] T c8f16f881d034a7bbafc7f08e1d80599 P 3203cbbffdea4401b85aea8372493be4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:37:07.070611 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 861
I20260929 10:37:07.070765 1293 raft_consensus.cc:2258] T c8f16f881d034a7bbafc7f08e1d80599 P 8a054ab9a11c412bad3e1afa16a950ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:37:07.071151 1292 tablet_replica.cc:330] T c8f16f881d034a7bbafc7f08e1d80599 P 5d64a778ffa44370931238b948061bad: stopping tablet replica
I20260929 10:37:07.071805 1292 raft_consensus.cc:2229] T c8f16f881d034a7bbafc7f08e1d80599 P 5d64a778ffa44370931238b948061bad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:37:07.072093 1292 raft_consensus.cc:2258] T c8f16f881d034a7bbafc7f08e1d80599 P 5d64a778ffa44370931238b948061bad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:37:07.073923 1292 ts_tablet_manager.cc:1905] T c8f16f881d034a7bbafc7f08e1d80599 P 5d64a778ffa44370931238b948061bad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:37:07.074605 1293 ts_tablet_manager.cc:1905] T c8f16f881d034a7bbafc7f08e1d80599 P 8a054ab9a11c412bad3e1afa16a950ce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:37:07.077288 1293 ts_tablet_manager.cc:1918] T c8f16f881d034a7bbafc7f08e1d80599 P 8a054ab9a11c412bad3e1afa16a950ce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:37:07.077448 1293 log.cc:1196] T c8f16f881d034a7bbafc7f08e1d80599 P 8a054ab9a11c412bad3e1afa16a950ce: Deleting WAL directory at /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790677981592896-15689-0/minicluster-data/ts-2/wal/wals/c8f16f881d034a7bbafc7f08e1d80599
I20260929 10:37:07.077747 1292 ts_tablet_manager.cc:1918] T c8f16f881d034a7bbafc7f08e1d80599 P 5d64a778ffa44370931238b948061bad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:37:07.077836 1293 ts_tablet_manager.cc:1939] T c8f16f881d034a7bbafc7f08e1d80599 P 8a054ab9a11c412bad3e1afa16a950ce: Deleting consensus metadata
I20260929 10:37:07.077935 1292 log.cc:1196] T c8f16f881d034a7bbafc7f08e1d80599 P 5d64a778ffa44370931238b948061bad: Deleting WAL directory at /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790677981592896-15689-0/minicluster-data/ts-1/wal/wals/c8f16f881d034a7bbafc7f08e1d80599
I20260929 10:37:07.078351 1292 ts_tablet_manager.cc:1939] T c8f16f881d034a7bbafc7f08e1d80599 P 5d64a778ffa44370931238b948061bad: Deleting consensus metadata
I20260929 10:37:07.079144 788 catalog_manager.cc:4841] TS 8a054ab9a11c412bad3e1afa16a950ce (127.15.82.67:43371): tablet c8f16f881d034a7bbafc7f08e1d80599 (table test-table [id=912e9883dbe74af2a93dbb0ddb4faf9c]) successfully deleted
W20260929 10:37:07.079542 785 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.82.65:34453 (error 108)
I20260929 10:37:07.080088 785 catalog_manager.cc:4841] TS 5d64a778ffa44370931238b948061bad (127.15.82.66:37137): tablet c8f16f881d034a7bbafc7f08e1d80599 (table test-table [id=912e9883dbe74af2a93dbb0ddb4faf9c]) successfully deleted
W20260929 10:37:07.080747 1265 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:37:07.080394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.65:34453 (local address 127.0.0.1:60924)
0929 10:37:07.080537 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:07.080619 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.65:34453: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":49}
I20260929 10:37:07.080921 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 992
W20260929 10:37:07.082444 785 catalog_manager.cc:4584] TS 3203cbbffdea4401b85aea8372493be4 (127.15.82.65:34453): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c8f16f881d034a7bbafc7f08e1d80599: Network error: Client connection negotiation failed: client connection to 127.15.82.65:34453: connect: Connection reset by peer (error 104)
I20260929 10:37:07.087466 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 1123
I20260929 10:37:07.092711 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 770
2026-09-29T10:37:07Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1143 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:37:07 dist-test-slave-vg89 krb5kdc[1297](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:37:07 dist-test-slave-vg89 krb5kdc[1297](info): set up 2 sockets
Sep 29 10:37:07 dist-test-slave-vg89 krb5kdc[1297](info): commencing operation
krb5kdc: starting...
W20260929 10:37:08.138640 15689 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Sep 29 10:37:08 dist-test-slave-vg89 krb5kdc[1297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678228, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:37:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:37:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.82.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab.
Entry for principal kudu/127.15.82.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab.
I20260929 10:37:08.265435 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:34933
--webserver_interface=127.15.82.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab
--principal=kudu/127.15.82.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:39395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.82.126:34933
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:37:08.375325 1310 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:37:08.375620 1310 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:37:08.375663 1310 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:37:08.379704 1310 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:37:08.379770 1310 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:37:08.379789 1310 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:37:08.379839 1310 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:37:08.379873 1310 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:37:08.385180 1310 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:39395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.82.126:34933
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab
--principal=kudu/127.15.82.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:34933
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.82.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1310
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:37:08.386307 1310 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:37:08.387395 1310 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:37:08.395833 1316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:08.395833 1315 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:08.395852 1318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:37:08.396147 1310 server_base.cc:1034] running on GCE node
I20260929 10:37:08.396766 1310 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:37:08.397876 1310 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:37:08.399067 1310 hybrid_clock.cc:648] HybridClock initialized: now 1790678228399048 us; error 30 us; skew 500 ppm
Sep 29 10:37:08 dist-test-slave-vg89 krb5kdc[1297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678228, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:37:08.401950 1310 init.cc:377] Logged in from keytab as kudu/127.15.82.126@KRBTEST.COM (short username kudu)
I20260929 10:37:08.402902 1310 webserver.cc:460] Webserver started at http://127.15.82.126:41321/ using document root <none> and password file <none>
I20260929 10:37:08.403481 1310 fs_manager.cc:359] Metadata directory not provided
I20260929 10:37:08.403569 1310 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:37:08.403790 1310 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:37:08.405846 1310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/minicluster-data/master-0/data/instance:
uuid: "fb37a3f761b046f89664bc6ce2222bb0"
format_stamp: "Formatted at 2026-09-29 10:37:08 on dist-test-slave-vg89"
server_key: "ea9a0b13940ac3b0ed7cf99efc5dea20"
server_key_iv: "b76fa5ef79594f8e7d5433059d5098b3"
server_key_version: "encryptionkey@0"
I20260929 10:37:08.406466 1310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/minicluster-data/master-0/wal/instance:
uuid: "fb37a3f761b046f89664bc6ce2222bb0"
format_stamp: "Formatted at 2026-09-29 10:37:08 on dist-test-slave-vg89"
server_key: "ea9a0b13940ac3b0ed7cf99efc5dea20"
server_key_iv: "b76fa5ef79594f8e7d5433059d5098b3"
server_key_version: "encryptionkey@0"
I20260929 10:37:08.410068 1310 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260929 10:37:08.412606 1325 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:08.413872 1310 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 10:37:08.414014 1310 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/minicluster-data/master-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/minicluster-data/master-0/wal
uuid: "fb37a3f761b046f89664bc6ce2222bb0"
format_stamp: "Formatted at 2026-09-29 10:37:08 on dist-test-slave-vg89"
server_key: "ea9a0b13940ac3b0ed7cf99efc5dea20"
server_key_iv: "b76fa5ef79594f8e7d5433059d5098b3"
server_key_version: "encryptionkey@0"
I20260929 10:37:08.414126 1310 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:37:08.448593 1310 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:37:08.452095 1310 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:37:08.452306 1310 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:37:08.459183 1310 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.126:34933
I20260929 10:37:08.459206 1377 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.126:34933 every 8 connection(s)
I20260929 10:37:08.460119 1310 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
I20260929 10:37:08.462224 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 1310
I20260929 10:37:08.462368 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/minicluster-data/master-0/wal/instance
I20260929 10:37:08.462715 15689 external_mini_cluster.cc:1428] Setting key c0b02139be20e99ac756d3b4d677c00a
I20260929 10:37:08.463230 1378 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:37:08.470609 1378 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb37a3f761b046f89664bc6ce2222bb0: Bootstrap starting.
Sep 29 10:37:08 dist-test-slave-vg89 krb5kdc[1297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678228, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:37:08.473093 1378 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fb37a3f761b046f89664bc6ce2222bb0: Neither blocks nor log segments found. Creating new log.
I20260929 10:37:08.473949 1378 log.cc:824] T 00000000000000000000000000000000 P fb37a3f761b046f89664bc6ce2222bb0: Log is configured to *not* fsync() on all Append() calls
I20260929 10:37:08.475837 1378 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb37a3f761b046f89664bc6ce2222bb0: No bootstrap required, opened a new log
I20260929 10:37:08.478533 1381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:08.464428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57534 (local address 127.15.82.126:34933)
0929 10:37:08.464842 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:08.464871 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:08.464914 (+ 43us) server_negotiation.cc:407] Connection header received
0929 10:37:08.465988 (+ 1074us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:08.466011 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:08.466469 (+ 458us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:08.466867 (+ 398us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:08.467556 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:08.469054 (+ 1498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:08.469984 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:08.470666 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:08.470824 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:08.473588 (+ 2764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:08.473595 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:08.473610 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:37:08.473641 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:08.475484 (+ 1843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:08.475921 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:08.475943 (+ 22us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:08.475949 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:08.476092 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:08.476430 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:08.476433 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:08.476434 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:08.476897 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:08.477040 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:08.477381 (+ 341us) server_negotiation.cc:299] Negotiation successful
0929 10:37:08.477717 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":117,"threads_started":1}
I20260929 10:37:08.479868 1378 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb37a3f761b046f89664bc6ce2222bb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb37a3f761b046f89664bc6ce2222bb0" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 34933 } }
I20260929 10:37:08.480142 1378 raft_consensus.cc:374] T 00000000000000000000000000000000 P fb37a3f761b046f89664bc6ce2222bb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:37:08.480208 1378 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb37a3f761b046f89664bc6ce2222bb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb37a3f761b046f89664bc6ce2222bb0, State: Initialized, Role: FOLLOWER
I20260929 10:37:08.480649 1378 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb37a3f761b046f89664bc6ce2222bb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb37a3f761b046f89664bc6ce2222bb0" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 34933 } }
I20260929 10:37:08.480805 1378 raft_consensus.cc:388] T 00000000000000000000000000000000 P fb37a3f761b046f89664bc6ce2222bb0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:37:08.480875 1378 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb37a3f761b046f89664bc6ce2222bb0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:37:08.480993 1378 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb37a3f761b046f89664bc6ce2222bb0 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:37:08.481864 1378 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb37a3f761b046f89664bc6ce2222bb0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb37a3f761b046f89664bc6ce2222bb0" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 34933 } }
I20260929 10:37:08.482206 1378 leader_election.cc:302] T 00000000000000000000000000000000 P fb37a3f761b046f89664bc6ce2222bb0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fb37a3f761b046f89664bc6ce2222bb0; no voters:
I20260929 10:37:08.482543 1378 leader_election.cc:288] T 00000000000000000000000000000000 P fb37a3f761b046f89664bc6ce2222bb0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:37:08.482661 1383 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fb37a3f761b046f89664bc6ce2222bb0 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:37:08.482925 1383 raft_consensus.cc:686] T 00000000000000000000000000000000 P fb37a3f761b046f89664bc6ce2222bb0 [term 1 LEADER]: Becoming Leader. State: Replica: fb37a3f761b046f89664bc6ce2222bb0, State: Running, Role: LEADER
I20260929 10:37:08.483564 1383 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb37a3f761b046f89664bc6ce2222bb0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb37a3f761b046f89664bc6ce2222bb0" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 34933 } }
I20260929 10:37:08.483832 1378 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb37a3f761b046f89664bc6ce2222bb0 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:37:08.485203 1384 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb37a3f761b046f89664bc6ce2222bb0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fb37a3f761b046f89664bc6ce2222bb0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb37a3f761b046f89664bc6ce2222bb0" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 34933 } } }
I20260929 10:37:08.485260 1385 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb37a3f761b046f89664bc6ce2222bb0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb37a3f761b046f89664bc6ce2222bb0. Latest consensus state: current_term: 1 leader_uuid: "fb37a3f761b046f89664bc6ce2222bb0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb37a3f761b046f89664bc6ce2222bb0" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 34933 } } }
I20260929 10:37:08.485371 1385 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb37a3f761b046f89664bc6ce2222bb0 [sys.catalog]: This master's current role is: LEADER
I20260929 10:37:08.485321 1384 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb37a3f761b046f89664bc6ce2222bb0 [sys.catalog]: This master's current role is: LEADER
W20260929 10:37:08.488407 1398 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fb37a3f761b046f89664bc6ce2222bb0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 10:37:08.488488 1398 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 10:37:08.488554 1394 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:37:08.489382 1394 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:37:08.494517 1394 catalog_manager.cc:1347] Generated new cluster ID: bd00b9663a7d4ae7aab2ebc85f69a45c
I20260929 10:37:08.494583 1394 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:37:08.514097 1394 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:37:08.515161 1394 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:37:08.522416 1394 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fb37a3f761b046f89664bc6ce2222bb0: Generated new TSK 0
I20260929 10:37:08.523264 1394 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.82.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab.
Entry for principal kudu/127.15.82.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab.
I20260929 10:37:08.557140 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.65:0
--local_ip_for_outbound_sockets=127.15.82.65
--webserver_interface=127.15.82.65
--webserver_port=0
--tserver_master_addrs=127.15.82.126:34933
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab
--principal=kudu/127.15.82.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:39395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:37:08.670260 1404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:37:08.670554 1404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:37:08.670599 1404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:37:08.674786 1404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:37:08.674844 1404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:37:08.674934 1404 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.65
I20260929 10:37:08.680198 1404 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:39395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab
--principal=kudu/127.15.82.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.82.65
--webserver_port=0
--tserver_master_addrs=127.15.82.126:34933
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1404
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.65
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:37:08.681273 1404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:37:08.682348 1404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:37:08.692402 1409 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:08.692395 1412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:08.692395 1410 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:37:08.692803 1404 server_base.cc:1034] running on GCE node
I20260929 10:37:08.693411 1404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:37:08.693998 1404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:37:08.695152 1404 hybrid_clock.cc:648] HybridClock initialized: now 1790678228695134 us; error 36 us; skew 500 ppm
Sep 29 10:37:08 dist-test-slave-vg89 krb5kdc[1297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678228, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:37:08.704761 1404 init.cc:377] Logged in from keytab as kudu/127.15.82.65@KRBTEST.COM (short username kudu)
I20260929 10:37:08.705693 1404 webserver.cc:460] Webserver started at http://127.15.82.65:33473/ using document root <none> and password file <none>
I20260929 10:37:08.706241 1404 fs_manager.cc:359] Metadata directory not provided
I20260929 10:37:08.706303 1404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:37:08.706547 1404 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:37:08.708606 1404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/minicluster-data/ts-0/data/instance:
uuid: "f6ad69e5fe1349f1b1aed217c0059dc7"
format_stamp: "Formatted at 2026-09-29 10:37:08 on dist-test-slave-vg89"
server_key: "3c5c73ef785bd9d40aa700fcb093c677"
server_key_iv: "9201f4d6068f99bd877ff0d1a64b8e0a"
server_key_version: "encryptionkey@0"
I20260929 10:37:08.709117 1404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/minicluster-data/ts-0/wal/instance:
uuid: "f6ad69e5fe1349f1b1aed217c0059dc7"
format_stamp: "Formatted at 2026-09-29 10:37:08 on dist-test-slave-vg89"
server_key: "3c5c73ef785bd9d40aa700fcb093c677"
server_key_iv: "9201f4d6068f99bd877ff0d1a64b8e0a"
server_key_version: "encryptionkey@0"
I20260929 10:37:08.712885 1404 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.005s
I20260929 10:37:08.715505 1419 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:08.716776 1404 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260929 10:37:08.716873 1404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/minicluster-data/ts-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/minicluster-data/ts-0/wal
uuid: "f6ad69e5fe1349f1b1aed217c0059dc7"
format_stamp: "Formatted at 2026-09-29 10:37:08 on dist-test-slave-vg89"
server_key: "3c5c73ef785bd9d40aa700fcb093c677"
server_key_iv: "9201f4d6068f99bd877ff0d1a64b8e0a"
server_key_version: "encryptionkey@0"
I20260929 10:37:08.716959 1404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:37:08.742717 1404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:37:08.746150 1404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:37:08.746309 1404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:37:08.746943 1404 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:37:08.747731 1404 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:37:08.747781 1404 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:08.747822 1404 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:37:08.747844 1404 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:08.756423 1404 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.65:41841
I20260929 10:37:08.756448 1532 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.65:41841 every 8 connection(s)
I20260929 10:37:08.757313 1404 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
I20260929 10:37:08.764494 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 1404
I20260929 10:37:08.764680 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/minicluster-data/ts-0/wal/instance
I20260929 10:37:08.765082 15689 external_mini_cluster.cc:1428] Setting key 167659c55271f3fe208d2ad69ab9ec5d
Sep 29 10:37:08 dist-test-slave-vg89 krb5kdc[1297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678228, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.65@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:37:08.773031 1381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:08.759315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.65:56987 (local address 127.15.82.126:34933)
0929 10:37:08.759470 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:08.759475 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:08.760368 (+ 893us) server_negotiation.cc:407] Connection header received
0929 10:37:08.761885 (+ 1517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:08.761888 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:08.761938 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:08.762031 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:08.763473 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:08.764064 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:08.765959 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:08.766315 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:08.766447 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:08.769463 (+ 3016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:08.769466 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:08.769468 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:37:08.769494 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:08.770974 (+ 1480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:08.771504 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:08.771508 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:08.771510 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:08.771593 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:08.771935 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:08.771940 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:08.771941 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:08.772125 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:08.772229 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:08.772716 (+ 487us) server_negotiation.cc:299] Negotiation successful
0929 10:37:08.772851 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":58}
I20260929 10:37:08.773842 1535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:08.759689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:34933 (local address 127.15.82.65:56987)
0929 10:37:08.760193 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:08.760246 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0929 10:37:08.761605 (+ 1359us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:37:08.762211 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:37:08.762220 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:37:08.762751 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:37:08.763312 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:08.763329 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:08.764229 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:08.764236 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:08.765805 (+ 1569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:08.765817 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:08.766539 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:08.766544 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:08.766826 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:08.767487 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:37:08.767511 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:37:08.769292 (+ 1781us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:37:08.771095 (+ 1803us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:37:08.771102 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:37:08.771117 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:37:08.771399 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:37:08.771688 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:37:08.771692 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:37:08.771694 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:37:08.771842 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:37:08.772227 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:37:08.772245 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:37:08.772487 (+ 242us) client_negotiation.cc:769] Sending connection context
0929 10:37:08.772676 (+ 189us) client_negotiation.cc:240] Negotiation successful
0929 10:37:08.773004 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":125,"threads_started":1}
I20260929 10:37:08.775112 1533 heartbeater.cc:347] Connected to a master server at 127.15.82.126:34933
I20260929 10:37:08.775369 1533 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.15.82.66@KRBTEST.COM; defaulting to no policy
I20260929 10:37:08.776062 1533 heartbeater.cc:511] Master 127.15.82.126:34933 requested a full tablet report, sending...
I20260929 10:37:08.777719 1342 ts_manager.cc:194] Registered new tserver with Master: f6ad69e5fe1349f1b1aed217c0059dc7 (127.15.82.65:41841)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.66@KRBTEST.COM" created.
I20260929 10:37:08.779009 1342 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.82.65@KRBTEST.COM'} at 127.15.82.65:56987
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab.
Entry for principal kudu/127.15.82.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab.
I20260929 10:37:08.794047 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.66:0
--local_ip_for_outbound_sockets=127.15.82.66
--webserver_interface=127.15.82.66
--webserver_port=0
--tserver_master_addrs=127.15.82.126:34933
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab
--principal=kudu/127.15.82.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:39395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:37:08.909301 1538 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:37:08.909610 1538 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:37:08.909686 1538 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:37:08.913900 1538 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:37:08.913960 1538 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:37:08.914019 1538 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.66
I20260929 10:37:08.919847 1538 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:39395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab
--principal=kudu/127.15.82.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.82.66
--webserver_port=0
--tserver_master_addrs=127.15.82.126:34933
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1538
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.66
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:37:08.920974 1538 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:37:08.922072 1538 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:37:08.932089 1546 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:08.932128 1544 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:08.932128 1543 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:37:08.932408 1538 server_base.cc:1034] running on GCE node
I20260929 10:37:08.932775 1538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:37:08.933396 1538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:37:08.934548 1538 hybrid_clock.cc:648] HybridClock initialized: now 1790678228934529 us; error 29 us; skew 500 ppm
Sep 29 10:37:08 dist-test-slave-vg89 krb5kdc[1297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678228, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:37:08.937333 1538 init.cc:377] Logged in from keytab as kudu/127.15.82.66@KRBTEST.COM (short username kudu)
I20260929 10:37:08.938203 1538 webserver.cc:460] Webserver started at http://127.15.82.66:42057/ using document root <none> and password file <none>
I20260929 10:37:08.938860 1538 fs_manager.cc:359] Metadata directory not provided
I20260929 10:37:08.938953 1538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:37:08.939189 1538 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:37:08.941370 1538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/minicluster-data/ts-1/data/instance:
uuid: "974fd3103af342e298875c9c0c8428ef"
format_stamp: "Formatted at 2026-09-29 10:37:08 on dist-test-slave-vg89"
server_key: "297eeb6d91f916724333c13fa1f45ca4"
server_key_iv: "d7ad3dd045410eb2601fe4b484c091fb"
server_key_version: "encryptionkey@0"
I20260929 10:37:08.941994 1538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/minicluster-data/ts-1/wal/instance:
uuid: "974fd3103af342e298875c9c0c8428ef"
format_stamp: "Formatted at 2026-09-29 10:37:08 on dist-test-slave-vg89"
server_key: "297eeb6d91f916724333c13fa1f45ca4"
server_key_iv: "d7ad3dd045410eb2601fe4b484c091fb"
server_key_version: "encryptionkey@0"
I20260929 10:37:08.946058 1538 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260929 10:37:08.948657 1553 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:08.949954 1538 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 10:37:08.950103 1538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/minicluster-data/ts-1/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/minicluster-data/ts-1/wal
uuid: "974fd3103af342e298875c9c0c8428ef"
format_stamp: "Formatted at 2026-09-29 10:37:08 on dist-test-slave-vg89"
server_key: "297eeb6d91f916724333c13fa1f45ca4"
server_key_iv: "d7ad3dd045410eb2601fe4b484c091fb"
server_key_version: "encryptionkey@0"
I20260929 10:37:08.950234 1538 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:37:08.964638 1538 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:37:08.968070 1538 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:37:08.968292 1538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:37:08.968925 1538 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:37:08.969802 1538 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:37:08.969874 1538 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:08.969949 1538 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:37:08.969996 1538 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:08.979617 1538 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.66:37109
I20260929 10:37:08.979637 1666 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.66:37109 every 8 connection(s)
I20260929 10:37:08.980609 1538 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
I20260929 10:37:08.980876 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 1538
I20260929 10:37:08.981019 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/minicluster-data/ts-1/wal/instance
I20260929 10:37:08.981380 15689 external_mini_cluster.cc:1428] Setting key 0354c147bbd33c586919eb158bde768e
Sep 29 10:37:08 dist-test-slave-vg89 krb5kdc[1297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678228, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.66@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.82.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.67@KRBTEST.COM" created.
I20260929 10:37:08.998145 1381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:08.982597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:36075 (local address 127.15.82.126:34933)
0929 10:37:08.982729 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:08.982733 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:08.983905 (+ 1172us) server_negotiation.cc:407] Connection header received
0929 10:37:08.986006 (+ 2101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:08.986010 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:08.986098 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:08.986205 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:08.987919 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:08.988569 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:08.990666 (+ 2097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:08.990889 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:08.990972 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:08.994083 (+ 3111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:08.994086 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:08.994089 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:37:08.994115 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:08.995732 (+ 1617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:08.996284 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:08.996287 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:08.996288 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:08.996405 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:08.996834 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:08.996840 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:08.996845 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:08.997107 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:08.997217 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:08.997747 (+ 530us) server_negotiation.cc:299] Negotiation successful
0929 10:37:08.997906 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260929 10:37:08.999455 1669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:08.983148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:34933 (local address 127.15.82.66:36075)
0929 10:37:08.983728 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:08.983811 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0929 10:37:08.985730 (+ 1919us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:37:08.986443 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:37:08.986453 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:37:08.986953 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:37:08.987714 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:08.987740 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:08.988716 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:08.988723 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:08.990532 (+ 1809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:08.990548 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:08.991072 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:08.991080 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:08.991322 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:08.991995 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:37:08.992020 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:37:08.993905 (+ 1885us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:37:08.995847 (+ 1942us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:37:08.995852 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:37:08.995866 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:37:08.996142 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:37:08.996488 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:37:08.996495 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:37:08.996500 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:37:08.996714 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:37:08.997223 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:37:08.997229 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:37:08.997488 (+ 259us) client_negotiation.cc:769] Sending connection context
0929 10:37:08.997659 (+ 171us) client_negotiation.cc:240] Negotiation successful
0929 10:37:08.997947 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":135,"threads_started":1}
I20260929 10:37:09.000752 1667 heartbeater.cc:347] Connected to a master server at 127.15.82.126:34933
I20260929 10:37:09.001001 1667 heartbeater.cc:464] Registering TS with master...
I20260929 10:37:09.001495 1667 heartbeater.cc:511] Master 127.15.82.126:34933 requested a full tablet report, sending...
I20260929 10:37:09.002620 1342 ts_manager.cc:194] Registered new tserver with Master: 974fd3103af342e298875c9c0c8428ef (127.15.82.66:37109)
I20260929 10:37:09.003197 1342 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.82.66@KRBTEST.COM'} at 127.15.82.66:36075
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab.
Entry for principal kudu/127.15.82.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab.
I20260929 10:37:09.013499 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.67:0
--local_ip_for_outbound_sockets=127.15.82.67
--webserver_interface=127.15.82.67
--webserver_port=0
--tserver_master_addrs=127.15.82.126:34933
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab
--principal=kudu/127.15.82.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:39395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:37:09.131403 1672 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:37:09.131696 1672 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:37:09.131755 1672 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:37:09.135988 1672 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:37:09.136066 1672 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:37:09.136133 1672 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.67
I20260929 10:37:09.141721 1672 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:39395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab
--principal=kudu/127.15.82.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.82.67
--webserver_port=0
--tserver_master_addrs=127.15.82.126:34933
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1672
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.67
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:37:09.142922 1672 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:37:09.143945 1672 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:37:09.153733 1680 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:09.153725 1677 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:37:09.154001 1672 server_base.cc:1034] running on GCE node
W20260929 10:37:09.153725 1678 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:37:09.154593 1672 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:37:09.155308 1672 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:37:09.156529 1672 hybrid_clock.cc:648] HybridClock initialized: now 1790678229156496 us; error 50 us; skew 500 ppm
Sep 29 10:37:09 dist-test-slave-vg89 krb5kdc[1297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678229, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:37:09.159716 1672 init.cc:377] Logged in from keytab as kudu/127.15.82.67@KRBTEST.COM (short username kudu)
I20260929 10:37:09.160763 1672 webserver.cc:460] Webserver started at http://127.15.82.67:46021/ using document root <none> and password file <none>
I20260929 10:37:09.161355 1672 fs_manager.cc:359] Metadata directory not provided
I20260929 10:37:09.161442 1672 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:37:09.161677 1672 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:37:09.163897 1672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/minicluster-data/ts-2/data/instance:
uuid: "08caa8320a1f42a491fd69a5ab9d26c4"
format_stamp: "Formatted at 2026-09-29 10:37:09 on dist-test-slave-vg89"
server_key: "9019fab831d357d8eda653d4d4e6f1a6"
server_key_iv: "0c672d4206c18a584b2d43d54afa1431"
server_key_version: "encryptionkey@0"
I20260929 10:37:09.164474 1672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/minicluster-data/ts-2/wal/instance:
uuid: "08caa8320a1f42a491fd69a5ab9d26c4"
format_stamp: "Formatted at 2026-09-29 10:37:09 on dist-test-slave-vg89"
server_key: "9019fab831d357d8eda653d4d4e6f1a6"
server_key_iv: "0c672d4206c18a584b2d43d54afa1431"
server_key_version: "encryptionkey@0"
I20260929 10:37:09.168371 1672 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 10:37:09.170958 1687 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:09.172228 1672 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 10:37:09.172370 1672 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/minicluster-data/ts-2/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/minicluster-data/ts-2/wal
uuid: "08caa8320a1f42a491fd69a5ab9d26c4"
format_stamp: "Formatted at 2026-09-29 10:37:09 on dist-test-slave-vg89"
server_key: "9019fab831d357d8eda653d4d4e6f1a6"
server_key_iv: "0c672d4206c18a584b2d43d54afa1431"
server_key_version: "encryptionkey@0"
I20260929 10:37:09.172482 1672 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:37:09.187000 1672 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:37:09.191214 1672 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:37:09.191424 1672 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:37:09.192027 1672 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:37:09.192878 1672 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:37:09.192924 1672 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:09.192986 1672 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:37:09.193017 1672 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:09.201699 1672 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.67:40609
I20260929 10:37:09.201759 1800 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.67:40609 every 8 connection(s)
I20260929 10:37:09.202711 1672 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
I20260929 10:37:09.210898 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 1672
I20260929 10:37:09.211045 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/minicluster-data/ts-2/wal/instance
I20260929 10:37:09.211345 15689 external_mini_cluster.cc:1428] Setting key ba33d0921bf97df2c78c79fefeccdb8c
Sep 29 10:37:09 dist-test-slave-vg89 krb5kdc[1297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678229, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.67@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:37:09.217170 1381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:09.204543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.67:54155 (local address 127.15.82.126:34933)
0929 10:37:09.204758 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:09.204763 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:09.205546 (+ 783us) server_negotiation.cc:407] Connection header received
0929 10:37:09.207133 (+ 1587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:09.207136 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:09.207195 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:09.207299 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:09.208510 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:09.209061 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:09.210219 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:09.210515 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:09.210623 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:09.213573 (+ 2950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:09.213576 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:09.213578 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:37:09.213602 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:09.215220 (+ 1618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:09.215724 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:09.215726 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:09.215728 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:09.215811 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:09.216150 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:09.216153 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:09.216154 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:09.216355 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:09.216444 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:09.216942 (+ 498us) server_negotiation.cc:299] Negotiation successful
0929 10:37:09.217049 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260929 10:37:09.218120 1803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:09.204934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:34933 (local address 127.15.82.67:54155)
0929 10:37:09.205390 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:09.205434 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 10:37:09.206917 (+ 1483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:37:09.207488 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:37:09.207495 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:37:09.207863 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:37:09.208359 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:09.208373 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:09.209188 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:09.209191 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:09.210088 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:09.210097 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:09.210697 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:09.210702 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:09.210871 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:09.211552 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:37:09.211568 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:37:09.213375 (+ 1807us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:37:09.215351 (+ 1976us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:37:09.215355 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:37:09.215368 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:37:09.215634 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:37:09.215894 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:37:09.215896 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:37:09.215897 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:37:09.216043 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:37:09.216454 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:37:09.216460 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:37:09.216694 (+ 234us) client_negotiation.cc:769] Sending connection context
0929 10:37:09.216894 (+ 200us) client_negotiation.cc:240] Negotiation successful
0929 10:37:09.217233 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":113,"threads_started":1}
I20260929 10:37:09.219322 1801 heartbeater.cc:347] Connected to a master server at 127.15.82.126:34933
I20260929 10:37:09.219556 1801 heartbeater.cc:464] Registering TS with master...
I20260929 10:37:09.220010 1801 heartbeater.cc:511] Master 127.15.82.126:34933 requested a full tablet report, sending...
I20260929 10:37:09.221066 1342 ts_manager.cc:194] Registered new tserver with Master: 08caa8320a1f42a491fd69a5ab9d26c4 (127.15.82.67:40609)
I20260929 10:37:09.221694 1342 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.82.67@KRBTEST.COM'} at 127.15.82.67:54155
I20260929 10:37:09.225561 15689 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 10:37:09.235680 1381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:09.227759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57546 (local address 127.15.82.126:34933)
0929 10:37:09.227891 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:09.227895 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:09.228071 (+ 176us) server_negotiation.cc:407] Connection header received
0929 10:37:09.228171 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:09.228173 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:09.228215 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:09.228291 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:09.229022 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:09.229536 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:09.230433 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:09.230696 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:09.230820 (+ 124us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:09.232406 (+ 1586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:09.232410 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:09.232412 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:37:09.232436 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:09.233909 (+ 1473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:09.234468 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:09.234471 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:09.234472 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:09.234548 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:09.234852 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:09.234854 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:09.234856 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:09.235053 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:09.235168 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:09.235467 (+ 299us) server_negotiation.cc:299] Negotiation successful
0929 10:37:09.235551 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 10:37:09.238747 1342 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57546:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 10:37:09.241828 1342 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:37:09.257659 1817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:09.252315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56738 (local address 127.15.82.66:37109)
0929 10:37:09.252662 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:09.252676 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:09.253062 (+ 386us) server_negotiation.cc:407] Connection header received
0929 10:37:09.253202 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:09.253208 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:09.253391 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:09.253551 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:37:09.253759 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:09.254653 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:09.256430 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:09.256994 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:09.257110 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:09.257260 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:09.257400 (+ 140us) server_negotiation.cc:299] Negotiation successful
0929 10:37:09.257493 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":121,"threads_started":1}
I20260929 10:37:09.257654 1816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:09.251861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.66:37109 (local address 127.0.0.1:56738)
0929 10:37:09.252944 (+ 1083us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:09.252958 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 10:37:09.253081 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:37:09.253534 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:37:09.253537 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:37:09.253555 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:37:09.253656 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:09.253663 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:09.254861 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:09.254864 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:09.256334 (+ 1470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:09.256343 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:09.257121 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:09.257125 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:09.257225 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:09.257261 (+ 36us) client_negotiation.cc:769] Sending connection context
0929 10:37:09.257372 (+ 111us) client_negotiation.cc:240] Negotiation successful
0929 10:37:09.257493 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1017,"mutex_wait_us":68,"thread_start_us":100,"threads_started":1}
I20260929 10:37:09.258077 1812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:09.251046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.65:41841 (local address 127.0.0.1:47272)
0929 10:37:09.251741 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:09.251794 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0929 10:37:09.251941 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:37:09.253217 (+ 1276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:37:09.253220 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:37:09.253236 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:37:09.253339 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:09.253345 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:09.254825 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:09.254832 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:09.256204 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:09.256215 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:09.257579 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:09.257583 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:09.257672 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:09.257692 (+ 20us) client_negotiation.cc:769] Sending connection context
0929 10:37:09.257780 (+ 88us) client_negotiation.cc:240] Negotiation successful
0929 10:37:09.257887 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"mutex_wait_us":69,"spinlock_wait_cycles":7040,"thread_start_us":118,"threads_started":1}
I20260929 10:37:09.258078 1814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:09.251313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.67:40609 (local address 127.0.0.1:42664)
0929 10:37:09.251739 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:09.251796 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0929 10:37:09.251942 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:37:09.252860 (+ 918us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:37:09.252866 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:37:09.252889 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:37:09.253052 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:09.253059 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:09.254691 (+ 1632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:09.254694 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:09.256359 (+ 1665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:09.256370 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:09.257466 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:09.257470 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:09.257575 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:09.257601 (+ 26us) client_negotiation.cc:769] Sending connection context
0929 10:37:09.257657 (+ 56us) client_negotiation.cc:240] Negotiation successful
0929 10:37:09.257888 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"mutex_wait_us":79,"spinlock_wait_cycles":6784,"thread_start_us":84,"threads_started":1}
I20260929 10:37:09.258908 1813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:09.251202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47272 (local address 127.15.82.65:41841)
0929 10:37:09.252637 (+ 1435us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:09.252651 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:09.252675 (+ 24us) server_negotiation.cc:407] Connection header received
0929 10:37:09.252751 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:09.252759 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:09.252946 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:09.253094 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:37:09.253462 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:09.254675 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:09.256344 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:09.257222 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:09.257344 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:09.257522 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:09.258303 (+ 781us) server_negotiation.cc:299] Negotiation successful
0929 10:37:09.258748 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":915,"thread_start_us":77,"threads_started":1}
I20260929 10:37:09.259042 1815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:09.251767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42664 (local address 127.15.82.67:40609)
0929 10:37:09.252343 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:09.252356 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:09.252382 (+ 26us) server_negotiation.cc:407] Connection header received
0929 10:37:09.252465 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:09.252470 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:09.252684 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:09.253190 (+ 506us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:37:09.253332 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:09.254557 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:09.256514 (+ 1957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:09.257324 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:09.258495 (+ 1171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:09.258657 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:09.258716 (+ 59us) server_negotiation.cc:299] Negotiation successful
0929 10:37:09.258807 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":472,"thread_start_us":167,"threads_started":1}
I20260929 10:37:09.260968 1601 tablet_service.cc:1467] Processing CreateTablet for tablet d79527c6d9884961b0ffd8545e3ae318 (DEFAULT_TABLE table=test-table [id=63c47acaaf1c48d3a5c256918f9dcf29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:37:09.261317 1467 tablet_service.cc:1467] Processing CreateTablet for tablet d79527c6d9884961b0ffd8545e3ae318 (DEFAULT_TABLE table=test-table [id=63c47acaaf1c48d3a5c256918f9dcf29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:37:09.262017 1601 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d79527c6d9884961b0ffd8545e3ae318. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:37:09.262369 1467 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d79527c6d9884961b0ffd8545e3ae318. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:37:09.263979 1735 tablet_service.cc:1467] Processing CreateTablet for tablet d79527c6d9884961b0ffd8545e3ae318 (DEFAULT_TABLE table=test-table [id=63c47acaaf1c48d3a5c256918f9dcf29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:37:09.264962 1735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d79527c6d9884961b0ffd8545e3ae318. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:37:09.268898 1818 tablet_bootstrap.cc:492] T d79527c6d9884961b0ffd8545e3ae318 P 974fd3103af342e298875c9c0c8428ef: Bootstrap starting.
I20260929 10:37:09.270776 1819 tablet_bootstrap.cc:492] T d79527c6d9884961b0ffd8545e3ae318 P f6ad69e5fe1349f1b1aed217c0059dc7: Bootstrap starting.
I20260929 10:37:09.271461 1818 tablet_bootstrap.cc:654] T d79527c6d9884961b0ffd8545e3ae318 P 974fd3103af342e298875c9c0c8428ef: Neither blocks nor log segments found. Creating new log.
I20260929 10:37:09.271832 1820 tablet_bootstrap.cc:492] T d79527c6d9884961b0ffd8545e3ae318 P 08caa8320a1f42a491fd69a5ab9d26c4: Bootstrap starting.
I20260929 10:37:09.272375 1818 log.cc:824] T d79527c6d9884961b0ffd8545e3ae318 P 974fd3103af342e298875c9c0c8428ef: Log is configured to *not* fsync() on all Append() calls
I20260929 10:37:09.272912 1819 tablet_bootstrap.cc:654] T d79527c6d9884961b0ffd8545e3ae318 P f6ad69e5fe1349f1b1aed217c0059dc7: Neither blocks nor log segments found. Creating new log.
I20260929 10:37:09.274004 1820 tablet_bootstrap.cc:654] T d79527c6d9884961b0ffd8545e3ae318 P 08caa8320a1f42a491fd69a5ab9d26c4: Neither blocks nor log segments found. Creating new log.
I20260929 10:37:09.274235 1819 log.cc:824] T d79527c6d9884961b0ffd8545e3ae318 P f6ad69e5fe1349f1b1aed217c0059dc7: Log is configured to *not* fsync() on all Append() calls
I20260929 10:37:09.274791 1818 tablet_bootstrap.cc:492] T d79527c6d9884961b0ffd8545e3ae318 P 974fd3103af342e298875c9c0c8428ef: No bootstrap required, opened a new log
I20260929 10:37:09.275007 1818 ts_tablet_manager.cc:1397] T d79527c6d9884961b0ffd8545e3ae318 P 974fd3103af342e298875c9c0c8428ef: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260929 10:37:09.275496 1820 log.cc:824] T d79527c6d9884961b0ffd8545e3ae318 P 08caa8320a1f42a491fd69a5ab9d26c4: Log is configured to *not* fsync() on all Append() calls
I20260929 10:37:09.276139 1819 tablet_bootstrap.cc:492] T d79527c6d9884961b0ffd8545e3ae318 P f6ad69e5fe1349f1b1aed217c0059dc7: No bootstrap required, opened a new log
I20260929 10:37:09.276341 1819 ts_tablet_manager.cc:1397] T d79527c6d9884961b0ffd8545e3ae318 P f6ad69e5fe1349f1b1aed217c0059dc7: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260929 10:37:09.277582 1820 tablet_bootstrap.cc:492] T d79527c6d9884961b0ffd8545e3ae318 P 08caa8320a1f42a491fd69a5ab9d26c4: No bootstrap required, opened a new log
I20260929 10:37:09.277802 1820 ts_tablet_manager.cc:1397] T d79527c6d9884961b0ffd8545e3ae318 P 08caa8320a1f42a491fd69a5ab9d26c4: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260929 10:37:09.279124 1818 raft_consensus.cc:348] T d79527c6d9884961b0ffd8545e3ae318 P 974fd3103af342e298875c9c0c8428ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ad69e5fe1349f1b1aed217c0059dc7" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 41841 } } peers { permanent_uuid: "974fd3103af342e298875c9c0c8428ef" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 37109 } } peers { permanent_uuid: "08caa8320a1f42a491fd69a5ab9d26c4" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 40609 } }
I20260929 10:37:09.279417 1818 raft_consensus.cc:374] T d79527c6d9884961b0ffd8545e3ae318 P 974fd3103af342e298875c9c0c8428ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:37:09.279482 1818 raft_consensus.cc:729] T d79527c6d9884961b0ffd8545e3ae318 P 974fd3103af342e298875c9c0c8428ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 974fd3103af342e298875c9c0c8428ef, State: Initialized, Role: FOLLOWER
I20260929 10:37:09.279729 1819 raft_consensus.cc:348] T d79527c6d9884961b0ffd8545e3ae318 P f6ad69e5fe1349f1b1aed217c0059dc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ad69e5fe1349f1b1aed217c0059dc7" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 41841 } } peers { permanent_uuid: "974fd3103af342e298875c9c0c8428ef" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 37109 } } peers { permanent_uuid: "08caa8320a1f42a491fd69a5ab9d26c4" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 40609 } }
I20260929 10:37:09.279990 1819 raft_consensus.cc:374] T d79527c6d9884961b0ffd8545e3ae318 P f6ad69e5fe1349f1b1aed217c0059dc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:37:09.280041 1819 raft_consensus.cc:729] T d79527c6d9884961b0ffd8545e3ae318 P f6ad69e5fe1349f1b1aed217c0059dc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6ad69e5fe1349f1b1aed217c0059dc7, State: Initialized, Role: FOLLOWER
I20260929 10:37:09.279958 1818 consensus_queue.cc:260] T d79527c6d9884961b0ffd8545e3ae318 P 974fd3103af342e298875c9c0c8428ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ad69e5fe1349f1b1aed217c0059dc7" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 41841 } } peers { permanent_uuid: "974fd3103af342e298875c9c0c8428ef" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 37109 } } peers { permanent_uuid: "08caa8320a1f42a491fd69a5ab9d26c4" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 40609 } }
I20260929 10:37:09.280462 1819 consensus_queue.cc:260] T d79527c6d9884961b0ffd8545e3ae318 P f6ad69e5fe1349f1b1aed217c0059dc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ad69e5fe1349f1b1aed217c0059dc7" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 41841 } } peers { permanent_uuid: "974fd3103af342e298875c9c0c8428ef" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 37109 } } peers { permanent_uuid: "08caa8320a1f42a491fd69a5ab9d26c4" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 40609 } }
I20260929 10:37:09.281014 1818 ts_tablet_manager.cc:1428] T d79527c6d9884961b0ffd8545e3ae318 P 974fd3103af342e298875c9c0c8428ef: Time spent starting tablet: real 0.006s user 0.002s sys 0.004s
I20260929 10:37:09.281359 1667 heartbeater.cc:503] Master 127.15.82.126:34933 was elected leader, sending a full tablet report...
I20260929 10:37:09.281392 1819 ts_tablet_manager.cc:1428] T d79527c6d9884961b0ffd8545e3ae318 P f6ad69e5fe1349f1b1aed217c0059dc7: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260929 10:37:09.281690 1533 heartbeater.cc:503] Master 127.15.82.126:34933 was elected leader, sending a full tablet report...
I20260929 10:37:09.282215 1820 raft_consensus.cc:348] T d79527c6d9884961b0ffd8545e3ae318 P 08caa8320a1f42a491fd69a5ab9d26c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ad69e5fe1349f1b1aed217c0059dc7" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 41841 } } peers { permanent_uuid: "974fd3103af342e298875c9c0c8428ef" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 37109 } } peers { permanent_uuid: "08caa8320a1f42a491fd69a5ab9d26c4" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 40609 } }
I20260929 10:37:09.282517 1820 raft_consensus.cc:374] T d79527c6d9884961b0ffd8545e3ae318 P 08caa8320a1f42a491fd69a5ab9d26c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:37:09.282579 1820 raft_consensus.cc:729] T d79527c6d9884961b0ffd8545e3ae318 P 08caa8320a1f42a491fd69a5ab9d26c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08caa8320a1f42a491fd69a5ab9d26c4, State: Initialized, Role: FOLLOWER
I20260929 10:37:09.283002 1820 consensus_queue.cc:260] T d79527c6d9884961b0ffd8545e3ae318 P 08caa8320a1f42a491fd69a5ab9d26c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ad69e5fe1349f1b1aed217c0059dc7" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 41841 } } peers { permanent_uuid: "974fd3103af342e298875c9c0c8428ef" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 37109 } } peers { permanent_uuid: "08caa8320a1f42a491fd69a5ab9d26c4" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 40609 } }
I20260929 10:37:09.283619 1801 heartbeater.cc:503] Master 127.15.82.126:34933 was elected leader, sending a full tablet report...
I20260929 10:37:09.283985 1820 ts_tablet_manager.cc:1428] T d79527c6d9884961b0ffd8545e3ae318 P 08caa8320a1f42a491fd69a5ab9d26c4: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
W20260929 10:37:09.453675 1802 tablet.cc:2369] T d79527c6d9884961b0ffd8545e3ae318 P 08caa8320a1f42a491fd69a5ab9d26c4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 10:37:09.481923 1668 tablet.cc:2369] T d79527c6d9884961b0ffd8545e3ae318 P 974fd3103af342e298875c9c0c8428ef: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 10:37:09.508730 1534 tablet.cc:2369] T d79527c6d9884961b0ffd8545e3ae318 P f6ad69e5fe1349f1b1aed217c0059dc7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 10:37:09.561486 1825 raft_consensus.cc:482] T d79527c6d9884961b0ffd8545e3ae318 P f6ad69e5fe1349f1b1aed217c0059dc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:37:09.561769 1825 raft_consensus.cc:504] T d79527c6d9884961b0ffd8545e3ae318 P f6ad69e5fe1349f1b1aed217c0059dc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ad69e5fe1349f1b1aed217c0059dc7" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 41841 } } peers { permanent_uuid: "974fd3103af342e298875c9c0c8428ef" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 37109 } } peers { permanent_uuid: "08caa8320a1f42a491fd69a5ab9d26c4" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 40609 } }
I20260929 10:37:09.563485 1825 leader_election.cc:288] T d79527c6d9884961b0ffd8545e3ae318 P f6ad69e5fe1349f1b1aed217c0059dc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 974fd3103af342e298875c9c0c8428ef (127.15.82.66:37109), 08caa8320a1f42a491fd69a5ab9d26c4 (127.15.82.67:40609)
I20260929 10:37:09.567682 1817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:09.564177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.65:55759 (local address 127.15.82.66:37109)
0929 10:37:09.564302 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:09.564307 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:09.564336 (+ 29us) server_negotiation.cc:407] Connection header received
0929 10:37:09.564503 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:09.564509 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:09.564651 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:09.564749 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:37:09.565040 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:09.565548 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:09.566741 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:09.567135 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:09.567262 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:09.567317 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:09.567490 (+ 173us) server_negotiation.cc:299] Negotiation successful
0929 10:37:09.567562 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260929 10:37:09.567698 1827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:09.563741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.66:37109 (local address 127.15.82.65:55759)
0929 10:37:09.564194 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:09.564215 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 10:37:09.564358 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:37:09.564756 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:37:09.564758 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:37:09.564773 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:37:09.564906 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:09.564912 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:09.565658 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:09.565664 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:09.566617 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:09.566626 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:09.567262 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:09.567266 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:09.567366 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:09.567388 (+ 22us) client_negotiation.cc:769] Sending connection context
0929 10:37:09.567474 (+ 86us) client_negotiation.cc:240] Negotiation successful
0929 10:37:09.567542 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":145,"threads_started":1}
I20260929 10:37:09.568203 1815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:09.563891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.65:35391 (local address 127.15.82.67:40609)
0929 10:37:09.564031 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:09.564036 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:09.564774 (+ 738us) server_negotiation.cc:407] Connection header received
0929 10:37:09.564874 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:09.564877 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:09.564931 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:09.565027 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:37:09.565336 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:09.565884 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:09.567119 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:09.567567 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:09.567700 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:09.567750 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:09.567972 (+ 222us) server_negotiation.cc:299] Negotiation successful
0929 10:37:09.568051 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260929 10:37:09.568291 1621 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d79527c6d9884961b0ffd8545e3ae318" candidate_uuid: "f6ad69e5fe1349f1b1aed217c0059dc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "974fd3103af342e298875c9c0c8428ef" is_pre_election: true
I20260929 10:37:09.568634 1828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:09.564113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.67:40609 (local address 127.15.82.65:35391)
0929 10:37:09.564478 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:09.564492 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 10:37:09.564598 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:37:09.565040 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:37:09.565043 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:37:09.565061 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:37:09.565205 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:09.565211 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:09.566009 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:09.566015 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:09.566989 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:09.566999 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:09.567691 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:09.567695 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:09.567803 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:09.567839 (+ 36us) client_negotiation.cc:769] Sending connection context
0929 10:37:09.568384 (+ 545us) client_negotiation.cc:240] Negotiation successful
0929 10:37:09.568483 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":91,"threads_started":1}
I20260929 10:37:09.568652 1621 raft_consensus.cc:2454] T d79527c6d9884961b0ffd8545e3ae318 P 974fd3103af342e298875c9c0c8428ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6ad69e5fe1349f1b1aed217c0059dc7 in term 0.
I20260929 10:37:09.569141 1423 leader_election.cc:302] T d79527c6d9884961b0ffd8545e3ae318 P f6ad69e5fe1349f1b1aed217c0059dc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 974fd3103af342e298875c9c0c8428ef, f6ad69e5fe1349f1b1aed217c0059dc7; no voters:
I20260929 10:37:09.569446 1825 raft_consensus.cc:2781] T d79527c6d9884961b0ffd8545e3ae318 P f6ad69e5fe1349f1b1aed217c0059dc7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:37:09.569552 1825 raft_consensus.cc:482] T d79527c6d9884961b0ffd8545e3ae318 P f6ad69e5fe1349f1b1aed217c0059dc7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:37:09.569612 1825 raft_consensus.cc:3037] T d79527c6d9884961b0ffd8545e3ae318 P f6ad69e5fe1349f1b1aed217c0059dc7 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:37:09.569787 1755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d79527c6d9884961b0ffd8545e3ae318" candidate_uuid: "f6ad69e5fe1349f1b1aed217c0059dc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08caa8320a1f42a491fd69a5ab9d26c4" is_pre_election: true
I20260929 10:37:09.570010 1755 raft_consensus.cc:2454] T d79527c6d9884961b0ffd8545e3ae318 P 08caa8320a1f42a491fd69a5ab9d26c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6ad69e5fe1349f1b1aed217c0059dc7 in term 0.
I20260929 10:37:09.570941 1825 raft_consensus.cc:504] T d79527c6d9884961b0ffd8545e3ae318 P f6ad69e5fe1349f1b1aed217c0059dc7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ad69e5fe1349f1b1aed217c0059dc7" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 41841 } } peers { permanent_uuid: "974fd3103af342e298875c9c0c8428ef" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 37109 } } peers { permanent_uuid: "08caa8320a1f42a491fd69a5ab9d26c4" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 40609 } }
I20260929 10:37:09.571306 1825 leader_election.cc:288] T d79527c6d9884961b0ffd8545e3ae318 P f6ad69e5fe1349f1b1aed217c0059dc7 [CANDIDATE]: Term 1 election: Requested vote from peers 974fd3103af342e298875c9c0c8428ef (127.15.82.66:37109), 08caa8320a1f42a491fd69a5ab9d26c4 (127.15.82.67:40609)
I20260929 10:37:09.571660 1621 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d79527c6d9884961b0ffd8545e3ae318" candidate_uuid: "f6ad69e5fe1349f1b1aed217c0059dc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "974fd3103af342e298875c9c0c8428ef"
I20260929 10:37:09.571717 1755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d79527c6d9884961b0ffd8545e3ae318" candidate_uuid: "f6ad69e5fe1349f1b1aed217c0059dc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08caa8320a1f42a491fd69a5ab9d26c4"
I20260929 10:37:09.571766 1621 raft_consensus.cc:3037] T d79527c6d9884961b0ffd8545e3ae318 P 974fd3103af342e298875c9c0c8428ef [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:37:09.571894 1755 raft_consensus.cc:3037] T d79527c6d9884961b0ffd8545e3ae318 P 08caa8320a1f42a491fd69a5ab9d26c4 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:37:09.572870 1621 raft_consensus.cc:2454] T d79527c6d9884961b0ffd8545e3ae318 P 974fd3103af342e298875c9c0c8428ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6ad69e5fe1349f1b1aed217c0059dc7 in term 1.
I20260929 10:37:09.573064 1755 raft_consensus.cc:2454] T d79527c6d9884961b0ffd8545e3ae318 P 08caa8320a1f42a491fd69a5ab9d26c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6ad69e5fe1349f1b1aed217c0059dc7 in term 1.
I20260929 10:37:09.573185 1423 leader_election.cc:302] T d79527c6d9884961b0ffd8545e3ae318 P f6ad69e5fe1349f1b1aed217c0059dc7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 974fd3103af342e298875c9c0c8428ef, f6ad69e5fe1349f1b1aed217c0059dc7; no voters:
I20260929 10:37:09.573390 1825 raft_consensus.cc:2781] T d79527c6d9884961b0ffd8545e3ae318 P f6ad69e5fe1349f1b1aed217c0059dc7 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:37:09.573665 1825 raft_consensus.cc:686] T d79527c6d9884961b0ffd8545e3ae318 P f6ad69e5fe1349f1b1aed217c0059dc7 [term 1 LEADER]: Becoming Leader. State: Replica: f6ad69e5fe1349f1b1aed217c0059dc7, State: Running, Role: LEADER
I20260929 10:37:09.574010 1825 consensus_queue.cc:237] T d79527c6d9884961b0ffd8545e3ae318 P f6ad69e5fe1349f1b1aed217c0059dc7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ad69e5fe1349f1b1aed217c0059dc7" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 41841 } } peers { permanent_uuid: "974fd3103af342e298875c9c0c8428ef" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 37109 } } peers { permanent_uuid: "08caa8320a1f42a491fd69a5ab9d26c4" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 40609 } }
I20260929 10:37:09.577239 1341 catalog_manager.cc:5495] T d79527c6d9884961b0ffd8545e3ae318 P f6ad69e5fe1349f1b1aed217c0059dc7 reported cstate change: term changed from 0 to 1, leader changed from <none> to f6ad69e5fe1349f1b1aed217c0059dc7 (127.15.82.65). New cstate: current_term: 1 leader_uuid: "f6ad69e5fe1349f1b1aed217c0059dc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ad69e5fe1349f1b1aed217c0059dc7" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "974fd3103af342e298875c9c0c8428ef" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 37109 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "08caa8320a1f42a491fd69a5ab9d26c4" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 40609 } health_report { overall_health: UNKNOWN } } }
I20260929 10:37:09.623636 1813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:09.619677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47282 (local address 127.15.82.65:41841)
0929 10:37:09.619808 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:09.619812 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:09.619827 (+ 15us) server_negotiation.cc:407] Connection header received
0929 10:37:09.619939 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:09.619941 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:09.619994 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:09.620114 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:37:09.620315 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:09.620856 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:09.621709 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:09.622140 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:09.622279 (+ 139us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:09.622694 (+ 415us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:37:09.623271 (+ 577us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:37:09.623391 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:09.623489 (+ 98us) server_negotiation.cc:299] Negotiation successful
0929 10:37:09.623523 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260929 10:37:09.629829 1621 raft_consensus.cc:1261] T d79527c6d9884961b0ffd8545e3ae318 P 974fd3103af342e298875c9c0c8428ef [term 1 FOLLOWER]: Refusing update from remote peer f6ad69e5fe1349f1b1aed217c0059dc7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:37:09.629829 1755 raft_consensus.cc:1261] T d79527c6d9884961b0ffd8545e3ae318 P 08caa8320a1f42a491fd69a5ab9d26c4 [term 1 FOLLOWER]: Refusing update from remote peer f6ad69e5fe1349f1b1aed217c0059dc7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:37:09.630573 1829 consensus_queue.cc:1035] T d79527c6d9884961b0ffd8545e3ae318 P f6ad69e5fe1349f1b1aed217c0059dc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "974fd3103af342e298875c9c0c8428ef" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 37109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:37:09.630764 1825 consensus_queue.cc:1035] T d79527c6d9884961b0ffd8545e3ae318 P f6ad69e5fe1349f1b1aed217c0059dc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "08caa8320a1f42a491fd69a5ab9d26c4" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 40609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:37:09.638058 1834 mvcc.cc:205] Tried to move back new op lower bound from 7334618028555943936 to 7334618028337901568. Current Snapshot: MvccSnapshot[applied={T|T < 7334618028555943936}]
I20260929 10:37:09.641429 1833 mvcc.cc:205] Tried to move back new op lower bound from 7334618028555943936 to 7334618028337901568. Current Snapshot: MvccSnapshot[applied={T|T < 7334618028555943936}]
I20260929 10:37:09.646787 1342 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57546:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:37:09.646955 1342 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57546:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:37:09.649514 1342 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fb37a3f761b046f89664bc6ce2222bb0: Sending DeleteTablet for 3 replicas of tablet d79527c6d9884961b0ffd8545e3ae318
I20260929 10:37:09.650379 1735 tablet_service.cc:1514] Processing DeleteTablet for tablet d79527c6d9884961b0ffd8545e3ae318 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:37:09 UTC) from {username='kudu', principal='kudu/127.15.82.126@KRBTEST.COM'} at 127.0.0.1:42664
I20260929 10:37:09.650597 1601 tablet_service.cc:1514] Processing DeleteTablet for tablet d79527c6d9884961b0ffd8545e3ae318 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:37:09 UTC) from {username='kudu', principal='kudu/127.15.82.126@KRBTEST.COM'} at 127.0.0.1:56738
I20260929 10:37:09.650592 1467 tablet_service.cc:1514] Processing DeleteTablet for tablet d79527c6d9884961b0ffd8545e3ae318 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:37:09 UTC) from {username='kudu', principal='kudu/127.15.82.126@KRBTEST.COM'} at 127.0.0.1:47272
I20260929 10:37:09.651098 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 1404
I20260929 10:37:09.652164 1843 tablet_replica.cc:330] T d79527c6d9884961b0ffd8545e3ae318 P 974fd3103af342e298875c9c0c8428ef: stopping tablet replica
I20260929 10:37:09.652529 1843 raft_consensus.cc:2229] T d79527c6d9884961b0ffd8545e3ae318 P 974fd3103af342e298875c9c0c8428ef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:37:09.652889 1843 raft_consensus.cc:2258] T d79527c6d9884961b0ffd8545e3ae318 P 974fd3103af342e298875c9c0c8428ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:37:09.654289 1843 ts_tablet_manager.cc:1905] T d79527c6d9884961b0ffd8545e3ae318 P 974fd3103af342e298875c9c0c8428ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:37:09.656665 1843 ts_tablet_manager.cc:1918] T d79527c6d9884961b0ffd8545e3ae318 P 974fd3103af342e298875c9c0c8428ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:37:09.656780 1843 log.cc:1196] T d79527c6d9884961b0ffd8545e3ae318 P 974fd3103af342e298875c9c0c8428ef: Deleting WAL directory at /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/minicluster-data/ts-1/wal/wals/d79527c6d9884961b0ffd8545e3ae318
I20260929 10:37:09.657096 1843 ts_tablet_manager.cc:1939] T d79527c6d9884961b0ffd8545e3ae318 P 974fd3103af342e298875c9c0c8428ef: Deleting consensus metadata
I20260929 10:37:09.657482 1842 tablet_replica.cc:330] T d79527c6d9884961b0ffd8545e3ae318 P 08caa8320a1f42a491fd69a5ab9d26c4: stopping tablet replica
I20260929 10:37:09.657830 1842 raft_consensus.cc:2229] T d79527c6d9884961b0ffd8545e3ae318 P 08caa8320a1f42a491fd69a5ab9d26c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:37:09.658171 1842 raft_consensus.cc:2258] T d79527c6d9884961b0ffd8545e3ae318 P 08caa8320a1f42a491fd69a5ab9d26c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:37:09.658528 1329 catalog_manager.cc:4841] TS 974fd3103af342e298875c9c0c8428ef (127.15.82.66:37109): tablet d79527c6d9884961b0ffd8545e3ae318 (table test-table [id=63c47acaaf1c48d3a5c256918f9dcf29]) successfully deleted
W20260929 10:37:09.658813 1329 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.82.65:41841 (error 108)
I20260929 10:37:09.659076 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 1538
I20260929 10:37:09.659785 1842 ts_tablet_manager.cc:1905] T d79527c6d9884961b0ffd8545e3ae318 P 08caa8320a1f42a491fd69a5ab9d26c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:37:09.661311 1814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:09.660945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.65:41841 (local address 127.0.0.1:47290)
0929 10:37:09.661126 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:09.661204 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.65:41841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
W20260929 10:37:09.661476 1329 catalog_manager.cc:4584] TS f6ad69e5fe1349f1b1aed217c0059dc7 (127.15.82.65:41841): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d79527c6d9884961b0ffd8545e3ae318: Network error: Client connection negotiation failed: client connection to 127.15.82.65:41841: connect: Connection refused (error 111)
I20260929 10:37:09.662101 1842 ts_tablet_manager.cc:1918] T d79527c6d9884961b0ffd8545e3ae318 P 08caa8320a1f42a491fd69a5ab9d26c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:37:09.662176 1842 log.cc:1196] T d79527c6d9884961b0ffd8545e3ae318 P 08caa8320a1f42a491fd69a5ab9d26c4: Deleting WAL directory at /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790677981592896-15689-0/minicluster-data/ts-2/wal/wals/d79527c6d9884961b0ffd8545e3ae318
I20260929 10:37:09.662554 1842 ts_tablet_manager.cc:1939] T d79527c6d9884961b0ffd8545e3ae318 P 08caa8320a1f42a491fd69a5ab9d26c4: Deleting consensus metadata
I20260929 10:37:09.663590 1328 catalog_manager.cc:4841] TS 08caa8320a1f42a491fd69a5ab9d26c4 (127.15.82.67:40609): tablet d79527c6d9884961b0ffd8545e3ae318 (table test-table [id=63c47acaaf1c48d3a5c256918f9dcf29]) successfully deleted
I20260929 10:37:09.665496 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 1672
I20260929 10:37:09.671020 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 1310
2026-09-29T10:37:09Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2578 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-29T10:37:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:37:09Z Disabled control of system clock
I20260929 10:37:09.702178 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790677981592896-15689-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790677981592896-15689-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790677981592896-15689-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:38209
--webserver_interface=127.15.82.126
--webserver_port=0
--builtin_ntp_servers=127.15.82.84:32875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.82.126:38209
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 10:37:09.813925 1849 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:37:09.814213 1849 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:37:09.814249 1849 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:37:09.818306 1849 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:37:09.818375 1849 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:37:09.818391 1849 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:37:09.818428 1849 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:37:09.818444 1849 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:37:09.823300 1849 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:32875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790677981592896-15689-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790677981592896-15689-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.82.126:38209
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:38209
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.82.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1849
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790677981592896-15689-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:37:09.824366 1849 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:37:09.825802 1849 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:37:09.834051 1857 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:09.834051 1855 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:09.834087 1854 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:37:09.834317 1849 server_base.cc:1034] running on GCE node
I20260929 10:37:09.834978 1849 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:37:09.836046 1849 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:37:09.837205 1849 hybrid_clock.cc:648] HybridClock initialized: now 1790678229837190 us; error 31 us; skew 500 ppm
I20260929 10:37:09.839140 1849 webserver.cc:460] Webserver started at http://127.15.82.126:42855/ using document root <none> and password file <none>
I20260929 10:37:09.839699 1849 fs_manager.cc:359] Metadata directory not provided
I20260929 10:37:09.839768 1849 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:37:09.839974 1849 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:37:09.842031 1849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790677981592896-15689-0/minicluster-data/master-0/data/instance:
uuid: "f5347d9271ef48dabd20511b67866429"
format_stamp: "Formatted at 2026-09-29 10:37:09 on dist-test-slave-vg89"
server_key: "2f42eafdf03f6a820ac8eafd4a3c5bba"
server_key_iv: "52684d4e76876c662915c00a8f9dbc7b"
server_key_version: "encryptionkey@0"
I20260929 10:37:09.842618 1849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790677981592896-15689-0/minicluster-data/master-0/wal/instance:
uuid: "f5347d9271ef48dabd20511b67866429"
format_stamp: "Formatted at 2026-09-29 10:37:09 on dist-test-slave-vg89"
server_key: "2f42eafdf03f6a820ac8eafd4a3c5bba"
server_key_iv: "52684d4e76876c662915c00a8f9dbc7b"
server_key_version: "encryptionkey@0"
I20260929 10:37:09.846350 1849 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260929 10:37:09.849097 1863 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:09.850553 1849 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 10:37:09.850682 1849 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790677981592896-15689-0/minicluster-data/master-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790677981592896-15689-0/minicluster-data/master-0/wal
uuid: "f5347d9271ef48dabd20511b67866429"
format_stamp: "Formatted at 2026-09-29 10:37:09 on dist-test-slave-vg89"
server_key: "2f42eafdf03f6a820ac8eafd4a3c5bba"
server_key_iv: "52684d4e76876c662915c00a8f9dbc7b"
server_key_version: "encryptionkey@0"
I20260929 10:37:09.850800 1849 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790677981592896-15689-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790677981592896-15689-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790677981592896-15689-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:37:09.861109 1849 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:37:09.861747 1849 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:37:09.861937 1849 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:37:09.868887 1849 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.126:38209
I20260929 10:37:09.868891 1915 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.126:38209 every 8 connection(s)
I20260929 10:37:09.869824 1849 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
I20260929 10:37:09.872861 1916 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:37:09.877938 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 1849
I20260929 10:37:09.878090 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790677981592896-15689-0/minicluster-data/master-0/wal/instance
I20260929 10:37:09.878389 15689 external_mini_cluster.cc:1428] Setting key 0568c0d7da1540a820e2c0d760167190
I20260929 10:37:09.878469 1916 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5347d9271ef48dabd20511b67866429: Bootstrap starting.
I20260929 10:37:09.880836 1916 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f5347d9271ef48dabd20511b67866429: Neither blocks nor log segments found. Creating new log.
I20260929 10:37:09.881600 1916 log.cc:824] T 00000000000000000000000000000000 P f5347d9271ef48dabd20511b67866429: Log is configured to *not* fsync() on all Append() calls
I20260929 10:37:09.883436 1916 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5347d9271ef48dabd20511b67866429: No bootstrap required, opened a new log
I20260929 10:37:09.887092 1916 raft_consensus.cc:348] T 00000000000000000000000000000000 P f5347d9271ef48dabd20511b67866429 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5347d9271ef48dabd20511b67866429" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 38209 } }
I20260929 10:37:09.887359 1916 raft_consensus.cc:374] T 00000000000000000000000000000000 P f5347d9271ef48dabd20511b67866429 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:37:09.887425 1916 raft_consensus.cc:729] T 00000000000000000000000000000000 P f5347d9271ef48dabd20511b67866429 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5347d9271ef48dabd20511b67866429, State: Initialized, Role: FOLLOWER
I20260929 10:37:09.887459 1919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:09.879794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41866 (local address 127.15.82.126:38209)
0929 10:37:09.880371 (+ 577us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:09.880385 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:09.880418 (+ 33us) server_negotiation.cc:407] Connection header received
0929 10:37:09.881123 (+ 705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:09.881141 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:09.881526 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:09.881855 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:09.882267 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:09.883271 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:09.884196 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:09.884664 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:09.884859 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:09.885713 (+ 854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:09.885717 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:09.885729 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:37:09.886094 (+ 365us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:09.886146 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:37:09.886164 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:09.886267 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:09.886446 (+ 179us) server_negotiation.cc:299] Negotiation successful
0929 10:37:09.886685 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":120,"threads_started":1}
I20260929 10:37:09.887918 1916 consensus_queue.cc:260] T 00000000000000000000000000000000 P f5347d9271ef48dabd20511b67866429 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5347d9271ef48dabd20511b67866429" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 38209 } }
I20260929 10:37:09.888042 1916 raft_consensus.cc:388] T 00000000000000000000000000000000 P f5347d9271ef48dabd20511b67866429 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:37:09.888123 1916 raft_consensus.cc:482] T 00000000000000000000000000000000 P f5347d9271ef48dabd20511b67866429 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:37:09.888230 1916 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f5347d9271ef48dabd20511b67866429 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:37:09.889075 1916 raft_consensus.cc:504] T 00000000000000000000000000000000 P f5347d9271ef48dabd20511b67866429 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5347d9271ef48dabd20511b67866429" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 38209 } }
I20260929 10:37:09.889405 1916 leader_election.cc:302] T 00000000000000000000000000000000 P f5347d9271ef48dabd20511b67866429 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f5347d9271ef48dabd20511b67866429; no voters:
I20260929 10:37:09.889714 1916 leader_election.cc:288] T 00000000000000000000000000000000 P f5347d9271ef48dabd20511b67866429 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:37:09.889776 1921 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f5347d9271ef48dabd20511b67866429 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:37:09.889984 1921 raft_consensus.cc:686] T 00000000000000000000000000000000 P f5347d9271ef48dabd20511b67866429 [term 1 LEADER]: Becoming Leader. State: Replica: f5347d9271ef48dabd20511b67866429, State: Running, Role: LEADER
I20260929 10:37:09.890266 1921 consensus_queue.cc:237] T 00000000000000000000000000000000 P f5347d9271ef48dabd20511b67866429 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5347d9271ef48dabd20511b67866429" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 38209 } }
I20260929 10:37:09.890990 1916 sys_catalog.cc:564] T 00000000000000000000000000000000 P f5347d9271ef48dabd20511b67866429 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:37:09.891927 1922 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5347d9271ef48dabd20511b67866429 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f5347d9271ef48dabd20511b67866429. Latest consensus state: current_term: 1 leader_uuid: "f5347d9271ef48dabd20511b67866429" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5347d9271ef48dabd20511b67866429" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 38209 } } }
I20260929 10:37:09.892053 1922 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5347d9271ef48dabd20511b67866429 [sys.catalog]: This master's current role is: LEADER
I20260929 10:37:09.892311 1923 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5347d9271ef48dabd20511b67866429 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f5347d9271ef48dabd20511b67866429" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5347d9271ef48dabd20511b67866429" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 38209 } } }
I20260929 10:37:09.892400 1923 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5347d9271ef48dabd20511b67866429 [sys.catalog]: This master's current role is: LEADER
I20260929 10:37:09.892513 1930 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:37:09.895604 1930 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:37:09.900866 1930 catalog_manager.cc:1347] Generated new cluster ID: 18d0bf53e7da4380853a5241686f6329
I20260929 10:37:09.900949 1930 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:37:09.942317 1930 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:37:09.943310 1930 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:37:09.949183 1930 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f5347d9271ef48dabd20511b67866429: Generated new TSK 0
I20260929 10:37:09.949942 1930 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 10:37:09.954807 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790677981592896-15689-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790677981592896-15689-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790677981592896-15689-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.65:0
--local_ip_for_outbound_sockets=127.15.82.65
--webserver_interface=127.15.82.65
--webserver_port=0
--tserver_master_addrs=127.15.82.126:38209
--builtin_ntp_servers=127.15.82.84:32875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 10:37:10.067428 1940 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:37:10.067704 1940 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:37:10.067763 1940 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:37:10.071888 1940 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:37:10.071941 1940 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:37:10.072036 1940 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.65
I20260929 10:37:10.076689 1940 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:32875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790677981592896-15689-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790677981592896-15689-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.82.65
--webserver_port=0
--tserver_master_addrs=127.15.82.126:38209
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1940
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.65
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790677981592896-15689-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:37:10.077751 1940 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:37:10.079212 1940 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:37:10.088620 1945 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:10.088627 1946 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:10.088627 1948 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:37:10.088848 1940 server_base.cc:1034] running on GCE node
I20260929 10:37:10.089278 1940 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:37:10.089905 1940 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:37:10.091054 1940 hybrid_clock.cc:648] HybridClock initialized: now 1790678230091021 us; error 45 us; skew 500 ppm
I20260929 10:37:10.093022 1940 webserver.cc:460] Webserver started at http://127.15.82.65:44799/ using document root <none> and password file <none>
I20260929 10:37:10.093580 1940 fs_manager.cc:359] Metadata directory not provided
I20260929 10:37:10.093640 1940 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:37:10.093883 1940 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:37:10.096086 1940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790677981592896-15689-0/minicluster-data/ts-0/data/instance:
uuid: "dd9d952166c045749e566e0b52d091dd"
format_stamp: "Formatted at 2026-09-29 10:37:10 on dist-test-slave-vg89"
server_key: "5c06b52249880bfed2a67207455ab74e"
server_key_iv: "3348711c3a130680ed8d1198be46fbda"
server_key_version: "encryptionkey@0"
I20260929 10:37:10.096629 1940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790677981592896-15689-0/minicluster-data/ts-0/wal/instance:
uuid: "dd9d952166c045749e566e0b52d091dd"
format_stamp: "Formatted at 2026-09-29 10:37:10 on dist-test-slave-vg89"
server_key: "5c06b52249880bfed2a67207455ab74e"
server_key_iv: "3348711c3a130680ed8d1198be46fbda"
server_key_version: "encryptionkey@0"
I20260929 10:37:10.100378 1940 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260929 10:37:10.102890 1954 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:10.104115 1940 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 10:37:10.104240 1940 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790677981592896-15689-0/minicluster-data/ts-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790677981592896-15689-0/minicluster-data/ts-0/wal
uuid: "dd9d952166c045749e566e0b52d091dd"
format_stamp: "Formatted at 2026-09-29 10:37:10 on dist-test-slave-vg89"
server_key: "5c06b52249880bfed2a67207455ab74e"
server_key_iv: "3348711c3a130680ed8d1198be46fbda"
server_key_version: "encryptionkey@0"
I20260929 10:37:10.104353 1940 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790677981592896-15689-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790677981592896-15689-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790677981592896-15689-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:37:10.139488 1940 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:37:10.140242 1940 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:37:10.140450 1940 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:37:10.141064 1940 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:37:10.142035 1940 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:37:10.142112 1940 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:10.142201 1940 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:37:10.142247 1940 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:10.151249 1940 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.65:38127
I20260929 10:37:10.151288 2067 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.65:38127 every 8 connection(s)
I20260929 10:37:10.152230 1940 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
I20260929 10:37:10.161518 1919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:10.154071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.65:38009 (local address 127.15.82.126:38209)
0929 10:37:10.154221 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:10.154225 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:10.155431 (+ 1206us) server_negotiation.cc:407] Connection header received
0929 10:37:10.156881 (+ 1450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:10.156884 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:10.156934 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:10.157015 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:10.157954 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:10.158593 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:10.159735 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:10.159989 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:10.160082 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:10.161113 (+ 1031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:10.161117 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:10.161119 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:37:10.161132 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:10.161151 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:37:10.161159 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:10.161218 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:10.161344 (+ 126us) server_negotiation.cc:299] Negotiation successful
0929 10:37:10.161397 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260929 10:37:10.161851 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 1940
I20260929 10:37:10.162003 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790677981592896-15689-0/minicluster-data/ts-0/wal/instance
I20260929 10:37:10.162297 15689 external_mini_cluster.cc:1428] Setting key 762c9f0863a221d4f88c582d6f709d64
I20260929 10:37:10.162566 2070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:10.154508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:38209 (local address 127.15.82.65:38009)
0929 10:37:10.155279 (+ 771us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:10.155323 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 10:37:10.156646 (+ 1323us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:37:10.157208 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:37:10.157215 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:37:10.157300 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:37:10.157794 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:10.157806 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:10.158699 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:10.158702 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:10.159602 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:10.159611 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:10.160135 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:10.160138 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:10.160283 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:10.160860 (+ 577us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 10:37:10.160861 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:37:10.160910 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 10:37:10.160974 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:37:10.161227 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:37:10.161234 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:37:10.161238 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 10:37:10.161331 (+ 93us) client_negotiation.cc:240] Negotiation successful
0929 10:37:10.161559 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":112,"threads_started":1}
I20260929 10:37:10.163758 2068 heartbeater.cc:347] Connected to a master server at 127.15.82.126:38209
I20260929 10:37:10.163998 2068 heartbeater.cc:464] Registering TS with master...
I20260929 10:37:10.164403 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790677981592896-15689-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790677981592896-15689-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790677981592896-15689-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.66:0
--local_ip_for_outbound_sockets=127.15.82.66
--webserver_interface=127.15.82.66
--webserver_port=0
--tserver_master_addrs=127.15.82.126:38209
--builtin_ntp_servers=127.15.82.84:32875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260929 10:37:10.164559 2068 heartbeater.cc:511] Master 127.15.82.126:38209 requested a full tablet report, sending...
I20260929 10:37:10.166138 1880 ts_manager.cc:194] Registered new tserver with Master: dd9d952166c045749e566e0b52d091dd (127.15.82.65:38127)
I20260929 10:37:10.167565 1880 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.82.65:38009
W20260929 10:37:10.278764 2071 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:37:10.279072 2071 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:37:10.279146 2071 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:37:10.283172 2071 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:37:10.283252 2071 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:37:10.283341 2071 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.66
I20260929 10:37:10.288111 2071 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:32875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790677981592896-15689-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790677981592896-15689-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.82.66
--webserver_port=0
--tserver_master_addrs=127.15.82.126:38209
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2071
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.66
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790677981592896-15689-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:37:10.289233 2071 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:37:10.290701 2071 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:37:10.300635 2076 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:10.300648 2079 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:37:10.300905 2071 server_base.cc:1034] running on GCE node
W20260929 10:37:10.301086 2077 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:37:10.301517 2071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:37:10.302160 2071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:37:10.303342 2071 hybrid_clock.cc:648] HybridClock initialized: now 1790678230303316 us; error 42 us; skew 500 ppm
I20260929 10:37:10.305132 2071 webserver.cc:460] Webserver started at http://127.15.82.66:40201/ using document root <none> and password file <none>
I20260929 10:37:10.305714 2071 fs_manager.cc:359] Metadata directory not provided
I20260929 10:37:10.305809 2071 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:37:10.306020 2071 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:37:10.308178 2071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790677981592896-15689-0/minicluster-data/ts-1/data/instance:
uuid: "12ba980a448b41119bf8b2088957ce88"
format_stamp: "Formatted at 2026-09-29 10:37:10 on dist-test-slave-vg89"
server_key: "22f1316a033fcd962ffbb512af6ca0fc"
server_key_iv: "13bd578134e6426c176d4491675c02de"
server_key_version: "encryptionkey@0"
I20260929 10:37:10.308746 2071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790677981592896-15689-0/minicluster-data/ts-1/wal/instance:
uuid: "12ba980a448b41119bf8b2088957ce88"
format_stamp: "Formatted at 2026-09-29 10:37:10 on dist-test-slave-vg89"
server_key: "22f1316a033fcd962ffbb512af6ca0fc"
server_key_iv: "13bd578134e6426c176d4491675c02de"
server_key_version: "encryptionkey@0"
I20260929 10:37:10.312463 2071 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260929 10:37:10.314922 2085 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:10.316866 2071 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 10:37:10.316987 2071 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790677981592896-15689-0/minicluster-data/ts-1/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790677981592896-15689-0/minicluster-data/ts-1/wal
uuid: "12ba980a448b41119bf8b2088957ce88"
format_stamp: "Formatted at 2026-09-29 10:37:10 on dist-test-slave-vg89"
server_key: "22f1316a033fcd962ffbb512af6ca0fc"
server_key_iv: "13bd578134e6426c176d4491675c02de"
server_key_version: "encryptionkey@0"
I20260929 10:37:10.317095 2071 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790677981592896-15689-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790677981592896-15689-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790677981592896-15689-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:37:10.327683 2071 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:37:10.328327 2071 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:37:10.328522 2071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:37:10.329064 2071 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:37:10.329955 2071 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:37:10.330003 2071 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:10.330040 2071 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:37:10.330093 2071 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:10.338668 2071 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.66:39043
I20260929 10:37:10.338706 2198 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.66:39043 every 8 connection(s)
I20260929 10:37:10.339557 2071 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
I20260929 10:37:10.342062 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 2071
I20260929 10:37:10.342191 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790677981592896-15689-0/minicluster-data/ts-1/wal/instance
I20260929 10:37:10.342573 15689 external_mini_cluster.cc:1428] Setting key 08db1b402915e7bc05d19f3885468ad6
I20260929 10:37:10.344703 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790677981592896-15689-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790677981592896-15689-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790677981592896-15689-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.67:0
--local_ip_for_outbound_sockets=127.15.82.67
--webserver_interface=127.15.82.67
--webserver_port=0
--tserver_master_addrs=127.15.82.126:38209
--builtin_ntp_servers=127.15.82.84:32875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260929 10:37:10.350564 1919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:10.341339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:51387 (local address 127.15.82.126:38209)
0929 10:37:10.341470 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:10.341474 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:10.342665 (+ 1191us) server_negotiation.cc:407] Connection header received
0929 10:37:10.344279 (+ 1614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:10.344283 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:10.344337 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:10.344422 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:10.345405 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:10.346269 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:10.348110 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:10.348336 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:10.348427 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:10.350014 (+ 1587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:10.350017 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:10.350020 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:37:10.350035 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:10.350062 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:37:10.350071 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:10.350146 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:10.350315 (+ 169us) server_negotiation.cc:299] Negotiation successful
0929 10:37:10.350383 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260929 10:37:10.351511 2201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:10.341743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:38209 (local address 127.15.82.66:51387)
0929 10:37:10.342491 (+ 748us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:10.342542 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0929 10:37:10.344066 (+ 1524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:37:10.344586 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:37:10.344593 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:37:10.344678 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:37:10.345231 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:10.345244 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:10.346445 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:10.346450 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:10.347968 (+ 1518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:10.347984 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:10.348501 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:10.348508 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:10.348787 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:10.349592 (+ 805us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 10:37:10.349596 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:37:10.349652 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 10:37:10.349746 (+ 94us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:37:10.350160 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:37:10.350170 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:37:10.350176 (+ 6us) client_negotiation.cc:769] Sending connection context
0929 10:37:10.350311 (+ 135us) client_negotiation.cc:240] Negotiation successful
0929 10:37:10.350735 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":145,"threads_started":1}
I20260929 10:37:10.352854 2199 heartbeater.cc:347] Connected to a master server at 127.15.82.126:38209
I20260929 10:37:10.353114 2199 heartbeater.cc:464] Registering TS with master...
I20260929 10:37:10.353690 2199 heartbeater.cc:511] Master 127.15.82.126:38209 requested a full tablet report, sending...
I20260929 10:37:10.354911 1880 ts_manager.cc:194] Registered new tserver with Master: 12ba980a448b41119bf8b2088957ce88 (127.15.82.66:39043)
I20260929 10:37:10.355571 1880 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.82.66:51387
W20260929 10:37:10.457715 2202 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:37:10.458000 2202 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:37:10.458067 2202 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:37:10.462145 2202 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:37:10.462218 2202 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:37:10.462304 2202 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.67
I20260929 10:37:10.466961 2202 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:32875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790677981592896-15689-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790677981592896-15689-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.82.67
--webserver_port=0
--tserver_master_addrs=127.15.82.126:38209
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2202
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.67
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790677981592896-15689-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:37:10.468009 2202 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:37:10.469385 2202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:37:10.478844 2210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:10.478864 2207 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:37:10.478910 2202 server_base.cc:1034] running on GCE node
W20260929 10:37:10.479009 2208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:37:10.479441 2202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:37:10.480103 2202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:37:10.481282 2202 hybrid_clock.cc:648] HybridClock initialized: now 1790678230481263 us; error 31 us; skew 500 ppm
I20260929 10:37:10.483144 2202 webserver.cc:460] Webserver started at http://127.15.82.67:40245/ using document root <none> and password file <none>
I20260929 10:37:10.483717 2202 fs_manager.cc:359] Metadata directory not provided
I20260929 10:37:10.483805 2202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:37:10.484016 2202 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:37:10.486068 2202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790677981592896-15689-0/minicluster-data/ts-2/data/instance:
uuid: "34ea5b69ae554789b1331cabfbfab691"
format_stamp: "Formatted at 2026-09-29 10:37:10 on dist-test-slave-vg89"
server_key: "e1adf98dcde5eadf1402b4a61a86f43f"
server_key_iv: "7b36e680c147bb538d9b74e4395cee66"
server_key_version: "encryptionkey@0"
I20260929 10:37:10.486645 2202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790677981592896-15689-0/minicluster-data/ts-2/wal/instance:
uuid: "34ea5b69ae554789b1331cabfbfab691"
format_stamp: "Formatted at 2026-09-29 10:37:10 on dist-test-slave-vg89"
server_key: "e1adf98dcde5eadf1402b4a61a86f43f"
server_key_iv: "7b36e680c147bb538d9b74e4395cee66"
server_key_version: "encryptionkey@0"
I20260929 10:37:10.490271 2202 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260929 10:37:10.492828 2216 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:10.494136 2202 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 10:37:10.494261 2202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790677981592896-15689-0/minicluster-data/ts-2/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790677981592896-15689-0/minicluster-data/ts-2/wal
uuid: "34ea5b69ae554789b1331cabfbfab691"
format_stamp: "Formatted at 2026-09-29 10:37:10 on dist-test-slave-vg89"
server_key: "e1adf98dcde5eadf1402b4a61a86f43f"
server_key_iv: "7b36e680c147bb538d9b74e4395cee66"
server_key_version: "encryptionkey@0"
I20260929 10:37:10.494367 2202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790677981592896-15689-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790677981592896-15689-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790677981592896-15689-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:37:10.512007 2202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:37:10.512641 2202 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:37:10.512827 2202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:37:10.513387 2202 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:37:10.514268 2202 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:37:10.514334 2202 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:10.514415 2202 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:37:10.514472 2202 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:10.523351 2202 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.67:43821
I20260929 10:37:10.523391 2329 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.67:43821 every 8 connection(s)
I20260929 10:37:10.524222 2202 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
I20260929 10:37:10.532100 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 2202
I20260929 10:37:10.532258 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790677981592896-15689-0/minicluster-data/ts-2/wal/instance
I20260929 10:37:10.532591 15689 external_mini_cluster.cc:1428] Setting key cb87d3a7e7cfc0f53e289e8c30acde15
I20260929 10:37:10.533406 1919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:10.526014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.67:53557 (local address 127.15.82.126:38209)
0929 10:37:10.526157 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:10.526161 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:10.527219 (+ 1058us) server_negotiation.cc:407] Connection header received
0929 10:37:10.528613 (+ 1394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:10.528617 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:10.528672 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:10.528777 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:10.529638 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:10.530169 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:10.531357 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:10.531588 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:10.531697 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:10.532853 (+ 1156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:10.532857 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:10.532859 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:37:10.532874 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:10.532899 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:37:10.532909 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:10.532981 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:10.533180 (+ 199us) server_negotiation.cc:299] Negotiation successful
0929 10:37:10.533260 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260929 10:37:10.534322 2332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:10.526384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:38209 (local address 127.15.82.67:53557)
0929 10:37:10.527067 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:10.527107 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0929 10:37:10.528405 (+ 1298us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:37:10.528950 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:37:10.528957 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:37:10.529046 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:37:10.529502 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:10.529514 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:10.530302 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:10.530310 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:10.531231 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:10.531240 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:10.531736 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:10.531740 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:10.531911 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:10.532548 (+ 637us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 10:37:10.532550 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:37:10.532589 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 10:37:10.532670 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:37:10.533001 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:37:10.533010 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:37:10.533015 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 10:37:10.533169 (+ 154us) client_negotiation.cc:240] Negotiation successful
0929 10:37:10.533475 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":106,"threads_started":1}
I20260929 10:37:10.535349 2330 heartbeater.cc:347] Connected to a master server at 127.15.82.126:38209
I20260929 10:37:10.535578 2330 heartbeater.cc:464] Registering TS with master...
I20260929 10:37:10.536088 2330 heartbeater.cc:511] Master 127.15.82.126:38209 requested a full tablet report, sending...
I20260929 10:37:10.537067 1880 ts_manager.cc:194] Registered new tserver with Master: 34ea5b69ae554789b1331cabfbfab691 (127.15.82.67:43821)
I20260929 10:37:10.537618 1880 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.82.67:53557
I20260929 10:37:10.546379 15689 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 10:37:10.552317 1919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:10.548404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41868 (local address 127.15.82.126:38209)
0929 10:37:10.548578 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:10.548583 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:10.548713 (+ 130us) server_negotiation.cc:407] Connection header received
0929 10:37:10.548856 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:10.548862 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:10.548906 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:10.548993 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:10.549275 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:10.549790 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:10.550689 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:10.550920 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:10.551024 (+ 104us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:10.551799 (+ 775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:10.551803 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:10.551805 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:37:10.551818 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:10.551842 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:37:10.551852 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:10.551988 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:10.552128 (+ 140us) server_negotiation.cc:299] Negotiation successful
0929 10:37:10.552169 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260929 10:37:10.555279 1880 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41868:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 10:37:10.558283 1880 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:37:10.573006 2346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:10.567789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33474 (local address 127.15.82.67:43821)
0929 10:37:10.568505 (+ 716us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:10.568519 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:10.568539 (+ 20us) server_negotiation.cc:407] Connection header received
0929 10:37:10.568605 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:10.568610 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:10.568777 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:10.569145 (+ 368us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:37:10.569244 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:10.569937 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:10.571644 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:10.572243 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:10.572380 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:10.572589 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:10.572694 (+ 105us) server_negotiation.cc:299] Negotiation successful
0929 10:37:10.572810 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":615,"thread_start_us":121,"threads_started":1}
I20260929 10:37:10.573012 2344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:10.567489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.67:43821 (local address 127.0.0.1:33474)
0929 10:37:10.568159 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:10.568171 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:37:10.568251 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:37:10.568924 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:37:10.568926 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:37:10.568941 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:37:10.569032 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:10.569038 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:10.570046 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:10.570052 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:10.571506 (+ 1454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:10.571518 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:10.572380 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:10.572384 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:10.572494 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:10.572536 (+ 42us) client_negotiation.cc:769] Sending connection context
0929 10:37:10.572667 (+ 131us) client_negotiation.cc:240] Negotiation successful
0929 10:37:10.572797 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":612,"mutex_wait_us":60,"thread_start_us":97,"threads_started":1}
I20260929 10:37:10.573179 2342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:10.566959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.65:38127 (local address 127.0.0.1:54478)
0929 10:37:10.567839 (+ 880us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:10.567854 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 10:37:10.567963 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:37:10.568593 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:37:10.568601 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:37:10.568625 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:37:10.568824 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:10.568833 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:10.570383 (+ 1550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:10.570387 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:10.571566 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:10.571597 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:10.572820 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:10.572823 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:10.572922 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:10.572944 (+ 22us) client_negotiation.cc:769] Sending connection context
0929 10:37:10.573009 (+ 65us) client_negotiation.cc:240] Negotiation successful
0929 10:37:10.573078 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":809,"thread_start_us":126,"threads_started":1}
I20260929 10:37:10.573024 2341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:10.566771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.66:39043 (local address 127.0.0.1:60444)
0929 10:37:10.567270 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:10.567318 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0929 10:37:10.567449 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:37:10.568778 (+ 1329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:37:10.568782 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:37:10.568800 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:37:10.568924 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:10.568930 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:10.570334 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:10.570338 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:10.571538 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:10.571548 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:10.572604 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:10.572608 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:10.572693 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:10.572716 (+ 23us) client_negotiation.cc:769] Sending connection context
0929 10:37:10.572767 (+ 51us) client_negotiation.cc:240] Negotiation successful
0929 10:37:10.572841 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":76,"threads_started":1}
I20260929 10:37:10.573597 2343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:10.566958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60444 (local address 127.15.82.66:39043)
0929 10:37:10.568240 (+ 1282us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:10.568254 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:10.568280 (+ 26us) server_negotiation.cc:407] Connection header received
0929 10:37:10.568363 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:10.568369 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:10.568581 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:10.568784 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:37:10.569021 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:10.570209 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:10.571666 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:10.572465 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:10.573087 (+ 622us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:10.573267 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:10.573321 (+ 54us) server_negotiation.cc:299] Negotiation successful
0929 10:37:10.573439 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1156,"thread_start_us":140,"threads_started":1}
I20260929 10:37:10.573800 2345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:10.567511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54478 (local address 127.15.82.65:38127)
0929 10:37:10.568065 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:10.568081 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:10.568108 (+ 27us) server_negotiation.cc:407] Connection header received
0929 10:37:10.568194 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:10.568202 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:10.568425 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:10.568590 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:37:10.568959 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:10.570217 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:10.571717 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:10.572464 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:10.572580 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:10.572766 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:10.573534 (+ 768us) server_negotiation.cc:299] Negotiation successful
0929 10:37:10.573639 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":119,"threads_started":1}
I20260929 10:37:10.576210 2133 tablet_service.cc:1467] Processing CreateTablet for tablet 6a9224da4bc24e78b8408eac6d8b4544 (DEFAULT_TABLE table=test-table [id=17149dd53da540f99be382ccdf4a62d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:37:10.576395 2264 tablet_service.cc:1467] Processing CreateTablet for tablet 6a9224da4bc24e78b8408eac6d8b4544 (DEFAULT_TABLE table=test-table [id=17149dd53da540f99be382ccdf4a62d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:37:10.576392 2002 tablet_service.cc:1467] Processing CreateTablet for tablet 6a9224da4bc24e78b8408eac6d8b4544 (DEFAULT_TABLE table=test-table [id=17149dd53da540f99be382ccdf4a62d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:37:10.577252 2133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a9224da4bc24e78b8408eac6d8b4544. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:37:10.577410 2264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a9224da4bc24e78b8408eac6d8b4544. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:37:10.577410 2002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a9224da4bc24e78b8408eac6d8b4544. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:37:10.583283 2347 tablet_bootstrap.cc:492] T 6a9224da4bc24e78b8408eac6d8b4544 P 12ba980a448b41119bf8b2088957ce88: Bootstrap starting.
I20260929 10:37:10.585441 2348 tablet_bootstrap.cc:492] T 6a9224da4bc24e78b8408eac6d8b4544 P dd9d952166c045749e566e0b52d091dd: Bootstrap starting.
I20260929 10:37:10.585440 2349 tablet_bootstrap.cc:492] T 6a9224da4bc24e78b8408eac6d8b4544 P 34ea5b69ae554789b1331cabfbfab691: Bootstrap starting.
I20260929 10:37:10.585568 2347 tablet_bootstrap.cc:654] T 6a9224da4bc24e78b8408eac6d8b4544 P 12ba980a448b41119bf8b2088957ce88: Neither blocks nor log segments found. Creating new log.
I20260929 10:37:10.586278 2347 log.cc:824] T 6a9224da4bc24e78b8408eac6d8b4544 P 12ba980a448b41119bf8b2088957ce88: Log is configured to *not* fsync() on all Append() calls
I20260929 10:37:10.587648 2348 tablet_bootstrap.cc:654] T 6a9224da4bc24e78b8408eac6d8b4544 P dd9d952166c045749e566e0b52d091dd: Neither blocks nor log segments found. Creating new log.
I20260929 10:37:10.587648 2349 tablet_bootstrap.cc:654] T 6a9224da4bc24e78b8408eac6d8b4544 P 34ea5b69ae554789b1331cabfbfab691: Neither blocks nor log segments found. Creating new log.
I20260929 10:37:10.588006 2347 tablet_bootstrap.cc:492] T 6a9224da4bc24e78b8408eac6d8b4544 P 12ba980a448b41119bf8b2088957ce88: No bootstrap required, opened a new log
I20260929 10:37:10.588196 2347 ts_tablet_manager.cc:1397] T 6a9224da4bc24e78b8408eac6d8b4544 P 12ba980a448b41119bf8b2088957ce88: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260929 10:37:10.588511 2348 log.cc:824] T 6a9224da4bc24e78b8408eac6d8b4544 P dd9d952166c045749e566e0b52d091dd: Log is configured to *not* fsync() on all Append() calls
I20260929 10:37:10.588511 2349 log.cc:824] T 6a9224da4bc24e78b8408eac6d8b4544 P 34ea5b69ae554789b1331cabfbfab691: Log is configured to *not* fsync() on all Append() calls
I20260929 10:37:10.590200 2348 tablet_bootstrap.cc:492] T 6a9224da4bc24e78b8408eac6d8b4544 P dd9d952166c045749e566e0b52d091dd: No bootstrap required, opened a new log
I20260929 10:37:10.590399 2348 ts_tablet_manager.cc:1397] T 6a9224da4bc24e78b8408eac6d8b4544 P dd9d952166c045749e566e0b52d091dd: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260929 10:37:10.591886 2349 tablet_bootstrap.cc:492] T 6a9224da4bc24e78b8408eac6d8b4544 P 34ea5b69ae554789b1331cabfbfab691: No bootstrap required, opened a new log
I20260929 10:37:10.591934 2347 raft_consensus.cc:348] T 6a9224da4bc24e78b8408eac6d8b4544 P 12ba980a448b41119bf8b2088957ce88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12ba980a448b41119bf8b2088957ce88" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 39043 } } peers { permanent_uuid: "dd9d952166c045749e566e0b52d091dd" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 38127 } } peers { permanent_uuid: "34ea5b69ae554789b1331cabfbfab691" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 43821 } }
I20260929 10:37:10.592103 2349 ts_tablet_manager.cc:1397] T 6a9224da4bc24e78b8408eac6d8b4544 P 34ea5b69ae554789b1331cabfbfab691: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260929 10:37:10.592183 2347 raft_consensus.cc:374] T 6a9224da4bc24e78b8408eac6d8b4544 P 12ba980a448b41119bf8b2088957ce88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:37:10.592262 2347 raft_consensus.cc:729] T 6a9224da4bc24e78b8408eac6d8b4544 P 12ba980a448b41119bf8b2088957ce88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12ba980a448b41119bf8b2088957ce88, State: Initialized, Role: FOLLOWER
I20260929 10:37:10.592762 2347 consensus_queue.cc:260] T 6a9224da4bc24e78b8408eac6d8b4544 P 12ba980a448b41119bf8b2088957ce88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12ba980a448b41119bf8b2088957ce88" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 39043 } } peers { permanent_uuid: "dd9d952166c045749e566e0b52d091dd" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 38127 } } peers { permanent_uuid: "34ea5b69ae554789b1331cabfbfab691" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 43821 } }
I20260929 10:37:10.593392 2199 heartbeater.cc:503] Master 127.15.82.126:38209 was elected leader, sending a full tablet report...
I20260929 10:37:10.593691 2347 ts_tablet_manager.cc:1428] T 6a9224da4bc24e78b8408eac6d8b4544 P 12ba980a448b41119bf8b2088957ce88: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260929 10:37:10.595016 2348 raft_consensus.cc:348] T 6a9224da4bc24e78b8408eac6d8b4544 P dd9d952166c045749e566e0b52d091dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12ba980a448b41119bf8b2088957ce88" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 39043 } } peers { permanent_uuid: "dd9d952166c045749e566e0b52d091dd" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 38127 } } peers { permanent_uuid: "34ea5b69ae554789b1331cabfbfab691" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 43821 } }
I20260929 10:37:10.595278 2348 raft_consensus.cc:374] T 6a9224da4bc24e78b8408eac6d8b4544 P dd9d952166c045749e566e0b52d091dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:37:10.595367 2348 raft_consensus.cc:729] T 6a9224da4bc24e78b8408eac6d8b4544 P dd9d952166c045749e566e0b52d091dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd9d952166c045749e566e0b52d091dd, State: Initialized, Role: FOLLOWER
I20260929 10:37:10.595825 2348 consensus_queue.cc:260] T 6a9224da4bc24e78b8408eac6d8b4544 P dd9d952166c045749e566e0b52d091dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12ba980a448b41119bf8b2088957ce88" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 39043 } } peers { permanent_uuid: "dd9d952166c045749e566e0b52d091dd" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 38127 } } peers { permanent_uuid: "34ea5b69ae554789b1331cabfbfab691" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 43821 } }
I20260929 10:37:10.596505 2068 heartbeater.cc:503] Master 127.15.82.126:38209 was elected leader, sending a full tablet report...
I20260929 10:37:10.596541 2349 raft_consensus.cc:348] T 6a9224da4bc24e78b8408eac6d8b4544 P 34ea5b69ae554789b1331cabfbfab691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12ba980a448b41119bf8b2088957ce88" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 39043 } } peers { permanent_uuid: "dd9d952166c045749e566e0b52d091dd" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 38127 } } peers { permanent_uuid: "34ea5b69ae554789b1331cabfbfab691" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 43821 } }
I20260929 10:37:10.596778 2349 raft_consensus.cc:374] T 6a9224da4bc24e78b8408eac6d8b4544 P 34ea5b69ae554789b1331cabfbfab691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:37:10.596832 2349 raft_consensus.cc:729] T 6a9224da4bc24e78b8408eac6d8b4544 P 34ea5b69ae554789b1331cabfbfab691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34ea5b69ae554789b1331cabfbfab691, State: Initialized, Role: FOLLOWER
I20260929 10:37:10.596868 2348 ts_tablet_manager.cc:1428] T 6a9224da4bc24e78b8408eac6d8b4544 P dd9d952166c045749e566e0b52d091dd: Time spent starting tablet: real 0.006s user 0.003s sys 0.002s
I20260929 10:37:10.597303 2349 consensus_queue.cc:260] T 6a9224da4bc24e78b8408eac6d8b4544 P 34ea5b69ae554789b1331cabfbfab691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12ba980a448b41119bf8b2088957ce88" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 39043 } } peers { permanent_uuid: "dd9d952166c045749e566e0b52d091dd" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 38127 } } peers { permanent_uuid: "34ea5b69ae554789b1331cabfbfab691" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 43821 } }
I20260929 10:37:10.598259 2330 heartbeater.cc:503] Master 127.15.82.126:38209 was elected leader, sending a full tablet report...
I20260929 10:37:10.598368 2349 ts_tablet_manager.cc:1428] T 6a9224da4bc24e78b8408eac6d8b4544 P 34ea5b69ae554789b1331cabfbfab691: Time spent starting tablet: real 0.006s user 0.000s sys 0.005s
W20260929 10:37:10.653385 2069 tablet.cc:2369] T 6a9224da4bc24e78b8408eac6d8b4544 P dd9d952166c045749e566e0b52d091dd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 10:37:10.775161 2331 tablet.cc:2369] T 6a9224da4bc24e78b8408eac6d8b4544 P 34ea5b69ae554789b1331cabfbfab691: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 10:37:10.776541 2354 raft_consensus.cc:482] T 6a9224da4bc24e78b8408eac6d8b4544 P dd9d952166c045749e566e0b52d091dd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:37:10.776780 2354 raft_consensus.cc:504] T 6a9224da4bc24e78b8408eac6d8b4544 P dd9d952166c045749e566e0b52d091dd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12ba980a448b41119bf8b2088957ce88" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 39043 } } peers { permanent_uuid: "dd9d952166c045749e566e0b52d091dd" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 38127 } } peers { permanent_uuid: "34ea5b69ae554789b1331cabfbfab691" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 43821 } }
I20260929 10:37:10.778067 2354 leader_election.cc:288] T 6a9224da4bc24e78b8408eac6d8b4544 P dd9d952166c045749e566e0b52d091dd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12ba980a448b41119bf8b2088957ce88 (127.15.82.66:39043), 34ea5b69ae554789b1331cabfbfab691 (127.15.82.67:43821)
I20260929 10:37:10.783386 2343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:10.778481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.65:41623 (local address 127.15.82.66:39043)
0929 10:37:10.778693 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:10.778698 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:10.778865 (+ 167us) server_negotiation.cc:407] Connection header received
0929 10:37:10.779055 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:10.779059 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:10.779112 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:10.779193 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:37:10.779553 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:10.780327 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:10.782044 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:10.782676 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:10.782836 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:10.782877 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:10.783180 (+ 303us) server_negotiation.cc:299] Negotiation successful
0929 10:37:10.783253 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260929 10:37:10.783386 2346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:10.778987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.65:47493 (local address 127.15.82.67:43821)
0929 10:37:10.779146 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:10.779153 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:10.779327 (+ 174us) server_negotiation.cc:407] Connection header received
0929 10:37:10.779425 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:10.779427 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:10.779467 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:10.779564 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:37:10.779833 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:10.780538 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:10.782044 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:10.782684 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:10.782813 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:10.782867 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:10.783182 (+ 315us) server_negotiation.cc:299] Negotiation successful
0929 10:37:10.783263 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260929 10:37:10.783464 2356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:10.778352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.66:39043 (local address 127.15.82.65:41623)
0929 10:37:10.778742 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:10.778764 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 10:37:10.778930 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:37:10.779243 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:37:10.779246 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:37:10.779275 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:37:10.779432 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:10.779440 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:10.780447 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:10.780450 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:10.781890 (+ 1440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:10.781903 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:10.782837 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:10.782840 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:10.783009 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:10.783061 (+ 52us) client_negotiation.cc:769] Sending connection context
0929 10:37:10.783153 (+ 92us) client_negotiation.cc:240] Negotiation successful
0929 10:37:10.783262 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"mutex_wait_us":123,"thread_start_us":170,"threads_started":1}
I20260929 10:37:10.783464 2357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:10.778862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.67:43821 (local address 127.15.82.65:47493)
0929 10:37:10.779227 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:10.779242 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 10:37:10.779336 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:37:10.779584 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:37:10.779587 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:37:10.779612 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:37:10.779723 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:10.779728 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:10.780632 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:10.780636 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:10.781918 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:10.781925 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:10.782818 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:10.782825 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:10.783020 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:10.783069 (+ 49us) client_negotiation.cc:769] Sending connection context
0929 10:37:10.783148 (+ 79us) client_negotiation.cc:240] Negotiation successful
0929 10:37:10.783267 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"mutex_wait_us":148,"thread_start_us":75,"threads_started":1}
I20260929 10:37:10.784196 2153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a9224da4bc24e78b8408eac6d8b4544" candidate_uuid: "dd9d952166c045749e566e0b52d091dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12ba980a448b41119bf8b2088957ce88" is_pre_election: true
I20260929 10:37:10.784201 2284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a9224da4bc24e78b8408eac6d8b4544" candidate_uuid: "dd9d952166c045749e566e0b52d091dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34ea5b69ae554789b1331cabfbfab691" is_pre_election: true
I20260929 10:37:10.784485 2153 raft_consensus.cc:2454] T 6a9224da4bc24e78b8408eac6d8b4544 P 12ba980a448b41119bf8b2088957ce88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd9d952166c045749e566e0b52d091dd in term 0.
I20260929 10:37:10.784502 2284 raft_consensus.cc:2454] T 6a9224da4bc24e78b8408eac6d8b4544 P 34ea5b69ae554789b1331cabfbfab691 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd9d952166c045749e566e0b52d091dd in term 0.
I20260929 10:37:10.784984 1958 leader_election.cc:302] T 6a9224da4bc24e78b8408eac6d8b4544 P dd9d952166c045749e566e0b52d091dd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12ba980a448b41119bf8b2088957ce88, dd9d952166c045749e566e0b52d091dd; no voters:
I20260929 10:37:10.785267 2354 raft_consensus.cc:2781] T 6a9224da4bc24e78b8408eac6d8b4544 P dd9d952166c045749e566e0b52d091dd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:37:10.785362 2354 raft_consensus.cc:482] T 6a9224da4bc24e78b8408eac6d8b4544 P dd9d952166c045749e566e0b52d091dd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:37:10.785420 2354 raft_consensus.cc:3037] T 6a9224da4bc24e78b8408eac6d8b4544 P dd9d952166c045749e566e0b52d091dd [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:37:10.786510 2354 raft_consensus.cc:504] T 6a9224da4bc24e78b8408eac6d8b4544 P dd9d952166c045749e566e0b52d091dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12ba980a448b41119bf8b2088957ce88" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 39043 } } peers { permanent_uuid: "dd9d952166c045749e566e0b52d091dd" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 38127 } } peers { permanent_uuid: "34ea5b69ae554789b1331cabfbfab691" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 43821 } }
I20260929 10:37:10.786875 2354 leader_election.cc:288] T 6a9224da4bc24e78b8408eac6d8b4544 P dd9d952166c045749e566e0b52d091dd [CANDIDATE]: Term 1 election: Requested vote from peers 12ba980a448b41119bf8b2088957ce88 (127.15.82.66:39043), 34ea5b69ae554789b1331cabfbfab691 (127.15.82.67:43821)
I20260929 10:37:10.787259 2284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a9224da4bc24e78b8408eac6d8b4544" candidate_uuid: "dd9d952166c045749e566e0b52d091dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34ea5b69ae554789b1331cabfbfab691"
I20260929 10:37:10.787259 2153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a9224da4bc24e78b8408eac6d8b4544" candidate_uuid: "dd9d952166c045749e566e0b52d091dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12ba980a448b41119bf8b2088957ce88"
I20260929 10:37:10.787386 2284 raft_consensus.cc:3037] T 6a9224da4bc24e78b8408eac6d8b4544 P 34ea5b69ae554789b1331cabfbfab691 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:37:10.787386 2153 raft_consensus.cc:3037] T 6a9224da4bc24e78b8408eac6d8b4544 P 12ba980a448b41119bf8b2088957ce88 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:37:10.788645 2284 raft_consensus.cc:2454] T 6a9224da4bc24e78b8408eac6d8b4544 P 34ea5b69ae554789b1331cabfbfab691 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd9d952166c045749e566e0b52d091dd in term 1.
I20260929 10:37:10.788645 2153 raft_consensus.cc:2454] T 6a9224da4bc24e78b8408eac6d8b4544 P 12ba980a448b41119bf8b2088957ce88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd9d952166c045749e566e0b52d091dd in term 1.
I20260929 10:37:10.788959 1958 leader_election.cc:302] T 6a9224da4bc24e78b8408eac6d8b4544 P dd9d952166c045749e566e0b52d091dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34ea5b69ae554789b1331cabfbfab691, dd9d952166c045749e566e0b52d091dd; no voters:
I20260929 10:37:10.789114 2354 raft_consensus.cc:2781] T 6a9224da4bc24e78b8408eac6d8b4544 P dd9d952166c045749e566e0b52d091dd [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:37:10.789317 2354 raft_consensus.cc:686] T 6a9224da4bc24e78b8408eac6d8b4544 P dd9d952166c045749e566e0b52d091dd [term 1 LEADER]: Becoming Leader. State: Replica: dd9d952166c045749e566e0b52d091dd, State: Running, Role: LEADER
I20260929 10:37:10.789675 2354 consensus_queue.cc:237] T 6a9224da4bc24e78b8408eac6d8b4544 P dd9d952166c045749e566e0b52d091dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12ba980a448b41119bf8b2088957ce88" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 39043 } } peers { permanent_uuid: "dd9d952166c045749e566e0b52d091dd" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 38127 } } peers { permanent_uuid: "34ea5b69ae554789b1331cabfbfab691" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 43821 } }
I20260929 10:37:10.792856 1880 catalog_manager.cc:5495] T 6a9224da4bc24e78b8408eac6d8b4544 P dd9d952166c045749e566e0b52d091dd reported cstate change: term changed from 0 to 1, leader changed from <none> to dd9d952166c045749e566e0b52d091dd (127.15.82.65). New cstate: current_term: 1 leader_uuid: "dd9d952166c045749e566e0b52d091dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12ba980a448b41119bf8b2088957ce88" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 39043 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd9d952166c045749e566e0b52d091dd" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 38127 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34ea5b69ae554789b1331cabfbfab691" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 43821 } health_report { overall_health: UNKNOWN } } }
I20260929 10:37:10.811189 2345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:10.807317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54490 (local address 127.15.82.65:38127)
0929 10:37:10.807426 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:10.807429 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:10.807463 (+ 34us) server_negotiation.cc:407] Connection header received
0929 10:37:10.807568 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:10.807570 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:10.807626 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:10.807713 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:37:10.807992 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:10.808567 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:10.809435 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:10.809651 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:10.809760 (+ 109us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:10.810185 (+ 425us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:37:10.810708 (+ 523us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:37:10.810830 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:10.810966 (+ 136us) server_negotiation.cc:299] Negotiation successful
0929 10:37:10.811022 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260929 10:37:10.817343 2153 raft_consensus.cc:1261] T 6a9224da4bc24e78b8408eac6d8b4544 P 12ba980a448b41119bf8b2088957ce88 [term 1 FOLLOWER]: Refusing update from remote peer dd9d952166c045749e566e0b52d091dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:37:10.817349 2284 raft_consensus.cc:1261] T 6a9224da4bc24e78b8408eac6d8b4544 P 34ea5b69ae554789b1331cabfbfab691 [term 1 FOLLOWER]: Refusing update from remote peer dd9d952166c045749e566e0b52d091dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:37:10.818109 2354 consensus_queue.cc:1035] T 6a9224da4bc24e78b8408eac6d8b4544 P dd9d952166c045749e566e0b52d091dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "12ba980a448b41119bf8b2088957ce88" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 39043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:37:10.818361 2358 consensus_queue.cc:1035] T 6a9224da4bc24e78b8408eac6d8b4544 P dd9d952166c045749e566e0b52d091dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "34ea5b69ae554789b1331cabfbfab691" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 43821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:37:10.835775 1879 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41868:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:37:10.835980 1879 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41868:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:37:10.838510 1879 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f5347d9271ef48dabd20511b67866429: Sending DeleteTablet for 3 replicas of tablet 6a9224da4bc24e78b8408eac6d8b4544
I20260929 10:37:10.839315 2002 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a9224da4bc24e78b8408eac6d8b4544 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:37:10 UTC) from {username='slave'} at 127.0.0.1:54478
I20260929 10:37:10.839510 2133 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a9224da4bc24e78b8408eac6d8b4544 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:37:10 UTC) from {username='slave'} at 127.0.0.1:60444
I20260929 10:37:10.839731 2264 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a9224da4bc24e78b8408eac6d8b4544 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:37:10 UTC) from {username='slave'} at 127.0.0.1:33474
I20260929 10:37:10.839804 2371 tablet_replica.cc:330] T 6a9224da4bc24e78b8408eac6d8b4544 P dd9d952166c045749e566e0b52d091dd: stopping tablet replica
I20260929 10:37:10.840001 2372 tablet_replica.cc:330] T 6a9224da4bc24e78b8408eac6d8b4544 P 12ba980a448b41119bf8b2088957ce88: stopping tablet replica
I20260929 10:37:10.840132 2371 raft_consensus.cc:2229] T 6a9224da4bc24e78b8408eac6d8b4544 P dd9d952166c045749e566e0b52d091dd [term 1 LEADER]: Raft consensus shutting down.
I20260929 10:37:10.840282 2372 raft_consensus.cc:2229] T 6a9224da4bc24e78b8408eac6d8b4544 P 12ba980a448b41119bf8b2088957ce88 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:37:10.840587 2372 raft_consensus.cc:2258] T 6a9224da4bc24e78b8408eac6d8b4544 P 12ba980a448b41119bf8b2088957ce88 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:37:10.840772 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 1940
I20260929 10:37:10.841727 2373 tablet_replica.cc:330] T 6a9224da4bc24e78b8408eac6d8b4544 P 34ea5b69ae554789b1331cabfbfab691: stopping tablet replica
I20260929 10:37:10.842013 2373 raft_consensus.cc:2229] T 6a9224da4bc24e78b8408eac6d8b4544 P 34ea5b69ae554789b1331cabfbfab691 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:37:10.842306 2373 raft_consensus.cc:2258] T 6a9224da4bc24e78b8408eac6d8b4544 P 34ea5b69ae554789b1331cabfbfab691 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:37:10.843415 2372 ts_tablet_manager.cc:1905] T 6a9224da4bc24e78b8408eac6d8b4544 P 12ba980a448b41119bf8b2088957ce88: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:37:10.843864 2373 ts_tablet_manager.cc:1905] T 6a9224da4bc24e78b8408eac6d8b4544 P 34ea5b69ae554789b1331cabfbfab691: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:37:10.845705 2372 ts_tablet_manager.cc:1918] T 6a9224da4bc24e78b8408eac6d8b4544 P 12ba980a448b41119bf8b2088957ce88: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:37:10.845809 2372 log.cc:1196] T 6a9224da4bc24e78b8408eac6d8b4544 P 12ba980a448b41119bf8b2088957ce88: Deleting WAL directory at /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790677981592896-15689-0/minicluster-data/ts-1/wal/wals/6a9224da4bc24e78b8408eac6d8b4544
I20260929 10:37:10.846091 2372 ts_tablet_manager.cc:1939] T 6a9224da4bc24e78b8408eac6d8b4544 P 12ba980a448b41119bf8b2088957ce88: Deleting consensus metadata
I20260929 10:37:10.847364 2373 ts_tablet_manager.cc:1918] T 6a9224da4bc24e78b8408eac6d8b4544 P 34ea5b69ae554789b1331cabfbfab691: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:37:10.847467 2373 log.cc:1196] T 6a9224da4bc24e78b8408eac6d8b4544 P 34ea5b69ae554789b1331cabfbfab691: Deleting WAL directory at /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790677981592896-15689-0/minicluster-data/ts-2/wal/wals/6a9224da4bc24e78b8408eac6d8b4544
I20260929 10:37:10.847456 1867 catalog_manager.cc:4841] TS 12ba980a448b41119bf8b2088957ce88 (127.15.82.66:39043): tablet 6a9224da4bc24e78b8408eac6d8b4544 (table test-table [id=17149dd53da540f99be382ccdf4a62d2]) successfully deleted
I20260929 10:37:10.847766 2373 ts_tablet_manager.cc:1939] T 6a9224da4bc24e78b8408eac6d8b4544 P 34ea5b69ae554789b1331cabfbfab691: Deleting consensus metadata
I20260929 10:37:10.849035 1867 catalog_manager.cc:4841] TS 34ea5b69ae554789b1331cabfbfab691 (127.15.82.67:43821): tablet 6a9224da4bc24e78b8408eac6d8b4544 (table test-table [id=17149dd53da540f99be382ccdf4a62d2]) successfully deleted
W20260929 10:37:10.849305 1864 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.82.65:38127 (error 108)
I20260929 10:37:10.850041 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 2071
I20260929 10:37:10.850283 2341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:10.849979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.65:38127 (local address 127.0.0.1:54502)
0929 10:37:10.850101 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:10.850188 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.65:38127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260929 10:37:10.850782 1864 catalog_manager.cc:4584] TS dd9d952166c045749e566e0b52d091dd (127.15.82.65:38127): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6a9224da4bc24e78b8408eac6d8b4544: Network error: Client connection negotiation failed: client connection to 127.15.82.65:38127: connect: Connection refused (error 111)
I20260929 10:37:10.856633 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 2202
I20260929 10:37:10.862282 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 1849
2026-09-29T10:37:10Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1190 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:37:10 dist-test-slave-vg89 krb5kdc[2377](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:37:10 dist-test-slave-vg89 krb5kdc[2377](info): set up 2 sockets
Sep 29 10:37:10 dist-test-slave-vg89 krb5kdc[2377](info): commencing operation
krb5kdc: starting...
W20260929 10:37:11.899296 15689 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Sep 29 10:37:11 dist-test-slave-vg89 krb5kdc[2377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678231, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:37:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:37:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.82.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab.
Entry for principal kudu/127.15.82.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab.
I20260929 10:37:12.027370 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:44079
--webserver_interface=127.15.82.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab
--principal=kudu/127.15.82.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:45285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.82.126:44079
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:37:12.136529 2390 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:37:12.136819 2390 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:37:12.136899 2390 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:37:12.140905 2390 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:37:12.140985 2390 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:37:12.141024 2390 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:37:12.141077 2390 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:37:12.141117 2390 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:37:12.146183 2390 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:45285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.82.126:44079
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab
--principal=kudu/127.15.82.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:44079
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.82.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2390
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:37:12.147230 2390 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:37:12.148243 2390 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:37:12.156697 2395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:12.156666 2398 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:12.156675 2396 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:37:12.156903 2390 server_base.cc:1034] running on GCE node
I20260929 10:37:12.157507 2390 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:37:12.158705 2390 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:37:12.159890 2390 hybrid_clock.cc:648] HybridClock initialized: now 1790678232159858 us; error 42 us; skew 500 ppm
Sep 29 10:37:12 dist-test-slave-vg89 krb5kdc[2377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678232, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:37:12.162868 2390 init.cc:377] Logged in from keytab as kudu/127.15.82.126@KRBTEST.COM (short username kudu)
I20260929 10:37:12.163648 2390 webserver.cc:460] Webserver started at http://127.15.82.126:34565/ using document root <none> and password file <none>
I20260929 10:37:12.164160 2390 fs_manager.cc:359] Metadata directory not provided
I20260929 10:37:12.164218 2390 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:37:12.164384 2390 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:37:12.166294 2390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/minicluster-data/master-0/data/instance:
uuid: "a7efcd953c6d4fafbb2a42b73eb2e950"
format_stamp: "Formatted at 2026-09-29 10:37:12 on dist-test-slave-vg89"
server_key: "bc9cc7f94050b4d7ea8b36dd55407d5d"
server_key_iv: "52167acc57f63cf7e5b2e887d9f3d773"
server_key_version: "encryptionkey@0"
I20260929 10:37:12.166862 2390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/minicluster-data/master-0/wal/instance:
uuid: "a7efcd953c6d4fafbb2a42b73eb2e950"
format_stamp: "Formatted at 2026-09-29 10:37:12 on dist-test-slave-vg89"
server_key: "bc9cc7f94050b4d7ea8b36dd55407d5d"
server_key_iv: "52167acc57f63cf7e5b2e887d9f3d773"
server_key_version: "encryptionkey@0"
I20260929 10:37:12.170365 2390 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260929 10:37:12.172740 2405 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:12.174003 2390 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 10:37:12.174153 2390 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/minicluster-data/master-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/minicluster-data/master-0/wal
uuid: "a7efcd953c6d4fafbb2a42b73eb2e950"
format_stamp: "Formatted at 2026-09-29 10:37:12 on dist-test-slave-vg89"
server_key: "bc9cc7f94050b4d7ea8b36dd55407d5d"
server_key_iv: "52167acc57f63cf7e5b2e887d9f3d773"
server_key_version: "encryptionkey@0"
I20260929 10:37:12.174281 2390 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:37:12.215050 2390 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:37:12.218031 2390 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:37:12.218242 2390 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:37:12.225303 2390 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.126:44079
I20260929 10:37:12.225407 2457 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.126:44079 every 8 connection(s)
I20260929 10:37:12.226300 2390 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
I20260929 10:37:12.229121 2458 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:37:12.234575 2458 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7efcd953c6d4fafbb2a42b73eb2e950: Bootstrap starting.
I20260929 10:37:12.235826 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 2390
I20260929 10:37:12.235956 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/minicluster-data/master-0/wal/instance
I20260929 10:37:12.236271 15689 external_mini_cluster.cc:1428] Setting key 96b6edd36a7a9efdc0a11cf77f6a5777
I20260929 10:37:12.237448 2458 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a7efcd953c6d4fafbb2a42b73eb2e950: Neither blocks nor log segments found. Creating new log.
I20260929 10:37:12.238225 2458 log.cc:824] T 00000000000000000000000000000000 P a7efcd953c6d4fafbb2a42b73eb2e950: Log is configured to *not* fsync() on all Append() calls
I20260929 10:37:12.240166 2458 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7efcd953c6d4fafbb2a42b73eb2e950: No bootstrap required, opened a new log
I20260929 10:37:12.243719 2458 raft_consensus.cc:348] T 00000000000000000000000000000000 P a7efcd953c6d4fafbb2a42b73eb2e950 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7efcd953c6d4fafbb2a42b73eb2e950" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 44079 } }
I20260929 10:37:12.243965 2458 raft_consensus.cc:374] T 00000000000000000000000000000000 P a7efcd953c6d4fafbb2a42b73eb2e950 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:37:12.244041 2458 raft_consensus.cc:729] T 00000000000000000000000000000000 P a7efcd953c6d4fafbb2a42b73eb2e950 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7efcd953c6d4fafbb2a42b73eb2e950, State: Initialized, Role: FOLLOWER
I20260929 10:37:12.244494 2458 consensus_queue.cc:260] T 00000000000000000000000000000000 P a7efcd953c6d4fafbb2a42b73eb2e950 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7efcd953c6d4fafbb2a42b73eb2e950" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 44079 } }
I20260929 10:37:12.244638 2458 raft_consensus.cc:388] T 00000000000000000000000000000000 P a7efcd953c6d4fafbb2a42b73eb2e950 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:37:12.244720 2458 raft_consensus.cc:482] T 00000000000000000000000000000000 P a7efcd953c6d4fafbb2a42b73eb2e950 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:37:12.244827 2458 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a7efcd953c6d4fafbb2a42b73eb2e950 [term 0 FOLLOWER]: Advancing to term 1
Sep 29 10:37:12 dist-test-slave-vg89 krb5kdc[2377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678231, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:37:12.245755 2458 raft_consensus.cc:504] T 00000000000000000000000000000000 P a7efcd953c6d4fafbb2a42b73eb2e950 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7efcd953c6d4fafbb2a42b73eb2e950" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 44079 } }
I20260929 10:37:12.246081 2458 leader_election.cc:302] T 00000000000000000000000000000000 P a7efcd953c6d4fafbb2a42b73eb2e950 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a7efcd953c6d4fafbb2a42b73eb2e950; no voters:
I20260929 10:37:12.246419 2458 leader_election.cc:288] T 00000000000000000000000000000000 P a7efcd953c6d4fafbb2a42b73eb2e950 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:37:12.246506 2463 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a7efcd953c6d4fafbb2a42b73eb2e950 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:37:12.246670 2463 raft_consensus.cc:686] T 00000000000000000000000000000000 P a7efcd953c6d4fafbb2a42b73eb2e950 [term 1 LEADER]: Becoming Leader. State: Replica: a7efcd953c6d4fafbb2a42b73eb2e950, State: Running, Role: LEADER
I20260929 10:37:12.246946 2463 consensus_queue.cc:237] T 00000000000000000000000000000000 P a7efcd953c6d4fafbb2a42b73eb2e950 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7efcd953c6d4fafbb2a42b73eb2e950" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 44079 } }
I20260929 10:37:12.247694 2458 sys_catalog.cc:564] T 00000000000000000000000000000000 P a7efcd953c6d4fafbb2a42b73eb2e950 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:37:12.248131 2464 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7efcd953c6d4fafbb2a42b73eb2e950 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a7efcd953c6d4fafbb2a42b73eb2e950. Latest consensus state: current_term: 1 leader_uuid: "a7efcd953c6d4fafbb2a42b73eb2e950" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7efcd953c6d4fafbb2a42b73eb2e950" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 44079 } } }
I20260929 10:37:12.248334 2464 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7efcd953c6d4fafbb2a42b73eb2e950 [sys.catalog]: This master's current role is: LEADER
I20260929 10:37:12.248576 2465 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7efcd953c6d4fafbb2a42b73eb2e950 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a7efcd953c6d4fafbb2a42b73eb2e950" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7efcd953c6d4fafbb2a42b73eb2e950" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 44079 } } }
I20260929 10:37:12.248755 2465 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7efcd953c6d4fafbb2a42b73eb2e950 [sys.catalog]: This master's current role is: LEADER
I20260929 10:37:12.248760 2468 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:37:12.251366 2461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:12.237433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47922 (local address 127.15.82.126:44079)
0929 10:37:12.237899 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:12.237914 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:12.237954 (+ 40us) server_negotiation.cc:407] Connection header received
0929 10:37:12.238765 (+ 811us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:12.238790 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:12.239182 (+ 392us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:12.239522 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:12.240349 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:12.241331 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:12.242183 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:12.242736 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:12.242908 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:12.245617 (+ 2709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:12.245628 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:12.245640 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:37:12.245666 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:12.247558 (+ 1892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:12.248081 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:12.248087 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:12.248095 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:12.248245 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:12.248596 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:12.248600 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:12.248601 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:12.249136 (+ 535us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:12.249325 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:12.250093 (+ 768us) server_negotiation.cc:299] Negotiation successful
0929 10:37:12.250405 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":134,"threads_started":1}
I20260929 10:37:12.252242 2468 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:37:12.257826 2468 catalog_manager.cc:1347] Generated new cluster ID: 07a6cb6f08f9459098291974c6e64d0b
I20260929 10:37:12.257899 2468 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:37:12.277747 2468 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:37:12.278959 2468 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:37:12.285650 2468 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a7efcd953c6d4fafbb2a42b73eb2e950: Generated new TSK 0
I20260929 10:37:12.286428 2468 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.82.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab.
Entry for principal kudu/127.15.82.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab.
I20260929 10:37:12.321424 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.65:0
--local_ip_for_outbound_sockets=127.15.82.65
--webserver_interface=127.15.82.65
--webserver_port=0
--tserver_master_addrs=127.15.82.126:44079
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab
--principal=kudu/127.15.82.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:45285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:37:12.437526 2484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:37:12.437769 2484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:37:12.437811 2484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:37:12.441875 2484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:37:12.441941 2484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:37:12.441996 2484 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.65
I20260929 10:37:12.447124 2484 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:45285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.65.keytab
--principal=kudu/127.15.82.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.82.65
--webserver_port=0
--tserver_master_addrs=127.15.82.126:44079
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2484
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.65
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:37:12.448169 2484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:37:12.449178 2484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:37:12.458671 2489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:12.458642 2492 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:12.458647 2490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:37:12.458868 2484 server_base.cc:1034] running on GCE node
I20260929 10:37:12.459261 2484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:37:12.459894 2484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:37:12.461079 2484 hybrid_clock.cc:648] HybridClock initialized: now 1790678232461049 us; error 47 us; skew 500 ppm
Sep 29 10:37:12 dist-test-slave-vg89 krb5kdc[2377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678232, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:37:12.463771 2484 init.cc:377] Logged in from keytab as kudu/127.15.82.65@KRBTEST.COM (short username kudu)
I20260929 10:37:12.464643 2484 webserver.cc:460] Webserver started at http://127.15.82.65:36155/ using document root <none> and password file <none>
I20260929 10:37:12.465193 2484 fs_manager.cc:359] Metadata directory not provided
I20260929 10:37:12.465281 2484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:37:12.465505 2484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:37:12.467579 2484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/minicluster-data/ts-0/data/instance:
uuid: "0df50643c6eb4749a0d587dbb8cf9132"
format_stamp: "Formatted at 2026-09-29 10:37:12 on dist-test-slave-vg89"
server_key: "8682031ccf2e668bde67b14abaae566e"
server_key_iv: "8627bdf664534cd28cb07fa7ed74f863"
server_key_version: "encryptionkey@0"
I20260929 10:37:12.468125 2484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/minicluster-data/ts-0/wal/instance:
uuid: "0df50643c6eb4749a0d587dbb8cf9132"
format_stamp: "Formatted at 2026-09-29 10:37:12 on dist-test-slave-vg89"
server_key: "8682031ccf2e668bde67b14abaae566e"
server_key_iv: "8627bdf664534cd28cb07fa7ed74f863"
server_key_version: "encryptionkey@0"
I20260929 10:37:12.471799 2484 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260929 10:37:12.474205 2499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:12.475471 2484 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 10:37:12.475620 2484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/minicluster-data/ts-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/minicluster-data/ts-0/wal
uuid: "0df50643c6eb4749a0d587dbb8cf9132"
format_stamp: "Formatted at 2026-09-29 10:37:12 on dist-test-slave-vg89"
server_key: "8682031ccf2e668bde67b14abaae566e"
server_key_iv: "8627bdf664534cd28cb07fa7ed74f863"
server_key_version: "encryptionkey@0"
I20260929 10:37:12.475733 2484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:37:12.499986 2484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:37:12.503068 2484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:37:12.503280 2484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:37:12.503911 2484 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:37:12.504747 2484 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:37:12.504817 2484 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:12.504890 2484 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:37:12.504930 2484 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:12.513609 2484 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.65:41389
I20260929 10:37:12.513633 2612 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.65:41389 every 8 connection(s)
I20260929 10:37:12.514592 2484 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
I20260929 10:37:12.518684 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 2484
I20260929 10:37:12.518831 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/minicluster-data/ts-0/wal/instance
I20260929 10:37:12.519107 15689 external_mini_cluster.cc:1428] Setting key aca82936e5044ca1f44d9b6090847c44
Sep 29 10:37:12 dist-test-slave-vg89 krb5kdc[2377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678232, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.65@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:37:12.529796 2461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:12.516427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.65:35177 (local address 127.15.82.126:44079)
0929 10:37:12.516559 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:12.516563 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:12.517393 (+ 830us) server_negotiation.cc:407] Connection header received
0929 10:37:12.518851 (+ 1458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:12.518855 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:12.518916 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:12.518994 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:12.520312 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:12.520891 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:12.522640 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:12.523050 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:12.523174 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:12.525877 (+ 2703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:12.525882 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:12.525886 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:37:12.525913 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:12.527669 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:12.528174 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:12.528179 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:12.528181 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:12.528280 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:12.528595 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:12.528599 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:12.528601 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:12.528862 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:12.528965 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:12.529463 (+ 498us) server_negotiation.cc:299] Negotiation successful
0929 10:37:12.529599 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
WARNING: no policy specified for kudu/127.15.82.66@KRBTEST.COM; defaulting to no policy
I20260929 10:37:12.530578 2615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:12.516811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:44079 (local address 127.15.82.65:35177)
0929 10:37:12.517255 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:12.517297 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 10:37:12.518627 (+ 1330us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:37:12.519184 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:37:12.519194 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:37:12.519554 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:37:12.520143 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:12.520157 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:12.521017 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:12.521023 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:12.522517 (+ 1494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:12.522528 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:12.523194 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:12.523196 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:12.523335 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:12.523930 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:37:12.523953 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:37:12.525709 (+ 1756us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:37:12.527793 (+ 2084us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:37:12.527798 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:37:12.527809 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:37:12.528075 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:37:12.528373 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:37:12.528375 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:37:12.528376 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:37:12.528506 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:37:12.528975 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:37:12.528981 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:37:12.529206 (+ 225us) client_negotiation.cc:769] Sending connection context
0929 10:37:12.529375 (+ 169us) client_negotiation.cc:240] Negotiation successful
0929 10:37:12.529629 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":121,"threads_started":1}
I20260929 10:37:12.531888 2613 heartbeater.cc:347] Connected to a master server at 127.15.82.126:44079
I20260929 10:37:12.532121 2613 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.66@KRBTEST.COM" created.
I20260929 10:37:12.532644 2613 heartbeater.cc:511] Master 127.15.82.126:44079 requested a full tablet report, sending...
I20260929 10:37:12.534327 2415 ts_manager.cc:194] Registered new tserver with Master: 0df50643c6eb4749a0d587dbb8cf9132 (127.15.82.65:41389)
I20260929 10:37:12.535869 2415 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.82.65@KRBTEST.COM'} at 127.15.82.65:35177
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab.
Entry for principal kudu/127.15.82.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab.
I20260929 10:37:12.550328 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.66:0
--local_ip_for_outbound_sockets=127.15.82.66
--webserver_interface=127.15.82.66
--webserver_port=0
--tserver_master_addrs=127.15.82.126:44079
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab
--principal=kudu/127.15.82.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:45285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:37:12.659948 2618 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:37:12.660230 2618 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:37:12.660307 2618 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:37:12.664247 2618 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:37:12.664323 2618 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:37:12.664407 2618 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.66
I20260929 10:37:12.669847 2618 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:45285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.66.keytab
--principal=kudu/127.15.82.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.82.66
--webserver_port=0
--tserver_master_addrs=127.15.82.126:44079
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2618
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.66
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:37:12.670919 2618 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:37:12.671940 2618 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:37:12.681726 2626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:12.681727 2624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:12.681730 2623 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:37:12.682304 2618 server_base.cc:1034] running on GCE node
I20260929 10:37:12.682758 2618 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:37:12.683396 2618 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:37:12.684584 2618 hybrid_clock.cc:648] HybridClock initialized: now 1790678232684564 us; error 31 us; skew 500 ppm
Sep 29 10:37:12 dist-test-slave-vg89 krb5kdc[2377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678232, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:37:12.687462 2618 init.cc:377] Logged in from keytab as kudu/127.15.82.66@KRBTEST.COM (short username kudu)
I20260929 10:37:12.688362 2618 webserver.cc:460] Webserver started at http://127.15.82.66:43289/ using document root <none> and password file <none>
I20260929 10:37:12.688941 2618 fs_manager.cc:359] Metadata directory not provided
I20260929 10:37:12.689000 2618 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:37:12.689229 2618 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:37:12.691308 2618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/minicluster-data/ts-1/data/instance:
uuid: "ae2547afea1f4ae1a8aef33eed4421a4"
format_stamp: "Formatted at 2026-09-29 10:37:12 on dist-test-slave-vg89"
server_key: "b4ae716053e9dcda19f20fb6302a3fd7"
server_key_iv: "d6e945601eb2517710c86bb8064b1b92"
server_key_version: "encryptionkey@0"
I20260929 10:37:12.691977 2618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/minicluster-data/ts-1/wal/instance:
uuid: "ae2547afea1f4ae1a8aef33eed4421a4"
format_stamp: "Formatted at 2026-09-29 10:37:12 on dist-test-slave-vg89"
server_key: "b4ae716053e9dcda19f20fb6302a3fd7"
server_key_iv: "d6e945601eb2517710c86bb8064b1b92"
server_key_version: "encryptionkey@0"
I20260929 10:37:12.695735 2618 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260929 10:37:12.698364 2633 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:12.699651 2618 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 10:37:12.699788 2618 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/minicluster-data/ts-1/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/minicluster-data/ts-1/wal
uuid: "ae2547afea1f4ae1a8aef33eed4421a4"
format_stamp: "Formatted at 2026-09-29 10:37:12 on dist-test-slave-vg89"
server_key: "b4ae716053e9dcda19f20fb6302a3fd7"
server_key_iv: "d6e945601eb2517710c86bb8064b1b92"
server_key_version: "encryptionkey@0"
I20260929 10:37:12.699915 2618 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:37:12.712740 2618 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:37:12.715785 2618 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:37:12.715987 2618 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:37:12.716797 2618 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:37:12.717610 2618 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:37:12.717682 2618 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:12.717784 2618 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:37:12.717835 2618 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:12.727756 2618 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.66:32915
I20260929 10:37:12.727820 2746 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.66:32915 every 8 connection(s)
I20260929 10:37:12.728685 2618 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
I20260929 10:37:12.737044 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 2618
I20260929 10:37:12.737197 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/minicluster-data/ts-1/wal/instance
I20260929 10:37:12.737504 15689 external_mini_cluster.cc:1428] Setting key 9e845b4a79c3f6f033d8259c1a0015fd
Sep 29 10:37:12 dist-test-slave-vg89 krb5kdc[2377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678232, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.66@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:37:12.743638 2461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:12.730543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:48287 (local address 127.15.82.126:44079)
0929 10:37:12.730681 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:12.730684 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:12.731516 (+ 832us) server_negotiation.cc:407] Connection header received
0929 10:37:12.732906 (+ 1390us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:12.732910 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:12.732969 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:12.733076 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:12.734232 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:12.734845 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:12.735996 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:12.736223 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:12.736350 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:12.739064 (+ 2714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:12.739092 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:12.739095 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:37:12.739125 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:12.741330 (+ 2205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:12.741877 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:12.741884 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:12.741885 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:12.741985 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:12.742442 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:12.742446 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:12.742448 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:12.742637 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:12.742763 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:12.743301 (+ 538us) server_negotiation.cc:299] Negotiation successful
0929 10:37:12.743435 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260929 10:37:12.744454 2749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:12.730904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:44079 (local address 127.15.82.66:48287)
0929 10:37:12.731357 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:12.731399 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 10:37:12.732671 (+ 1272us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:37:12.733247 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:37:12.733255 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:37:12.733603 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:37:12.734086 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:12.734099 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:12.734988 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:12.734991 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:12.735865 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:12.735874 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:12.736393 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:12.736397 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:12.736569 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:12.737203 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:37:12.737228 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:37:12.738895 (+ 1667us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:37:12.741461 (+ 2566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:37:12.741465 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:37:12.741477 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:37:12.741736 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:37:12.742127 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:37:12.742130 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:37:12.742132 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:37:12.742278 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:37:12.742762 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:37:12.742769 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:37:12.743062 (+ 293us) client_negotiation.cc:769] Sending connection context
0929 10:37:12.743257 (+ 195us) client_negotiation.cc:240] Negotiation successful
0929 10:37:12.743597 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":112,"threads_started":1}
I20260929 10:37:12.745853 2747 heartbeater.cc:347] Connected to a master server at 127.15.82.126:44079
I20260929 10:37:12.746197 2747 heartbeater.cc:464] Registering TS with master...
I20260929 10:37:12.746773 2747 heartbeater.cc:511] Master 127.15.82.126:44079 requested a full tablet report, sending...
I20260929 10:37:12.747962 2415 ts_manager.cc:194] Registered new tserver with Master: ae2547afea1f4ae1a8aef33eed4421a4 (127.15.82.66:32915)
WARNING: no policy specified for kudu/127.15.82.67@KRBTEST.COM; defaulting to no policy
I20260929 10:37:12.748687 2415 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.82.66@KRBTEST.COM'} at 127.15.82.66:48287
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab.
Entry for principal kudu/127.15.82.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab.
I20260929 10:37:12.767678 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.67:0
--local_ip_for_outbound_sockets=127.15.82.67
--webserver_interface=127.15.82.67
--webserver_port=0
--tserver_master_addrs=127.15.82.126:44079
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab
--principal=kudu/127.15.82.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:45285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:37:12.879927 2752 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:37:12.880177 2752 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:37:12.880247 2752 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:37:12.884269 2752 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:37:12.884351 2752 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:37:12.884438 2752 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.67
I20260929 10:37:12.889545 2752 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:45285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.67.keytab
--principal=kudu/127.15.82.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.82.67
--webserver_port=0
--tserver_master_addrs=127.15.82.126:44079
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2752
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.67
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:37:12.890542 2752 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:37:12.891505 2752 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 10:37:12.900774 2752 server_base.cc:1034] running on GCE node
W20260929 10:37:12.900689 2758 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:12.900689 2760 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:12.900693 2757 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:37:12.901500 2752 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:37:12.902086 2752 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:37:12.903257 2752 hybrid_clock.cc:648] HybridClock initialized: now 1790678232903240 us; error 27 us; skew 500 ppm
Sep 29 10:37:12 dist-test-slave-vg89 krb5kdc[2377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678232, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:37:12.905951 2752 init.cc:377] Logged in from keytab as kudu/127.15.82.67@KRBTEST.COM (short username kudu)
I20260929 10:37:12.906873 2752 webserver.cc:460] Webserver started at http://127.15.82.67:45789/ using document root <none> and password file <none>
I20260929 10:37:12.907407 2752 fs_manager.cc:359] Metadata directory not provided
I20260929 10:37:12.907493 2752 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:37:12.907716 2752 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:37:12.909657 2752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/minicluster-data/ts-2/data/instance:
uuid: "d06513b545e943768b9b5c7dccd5e209"
format_stamp: "Formatted at 2026-09-29 10:37:12 on dist-test-slave-vg89"
server_key: "64b002d3d9eeff2dfdbb4907797368cd"
server_key_iv: "6ad118e633f61d396c0c520f208c83d0"
server_key_version: "encryptionkey@0"
I20260929 10:37:12.910209 2752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/minicluster-data/ts-2/wal/instance:
uuid: "d06513b545e943768b9b5c7dccd5e209"
format_stamp: "Formatted at 2026-09-29 10:37:12 on dist-test-slave-vg89"
server_key: "64b002d3d9eeff2dfdbb4907797368cd"
server_key_iv: "6ad118e633f61d396c0c520f208c83d0"
server_key_version: "encryptionkey@0"
I20260929 10:37:12.913731 2752 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260929 10:37:12.916136 2767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:12.917418 2752 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260929 10:37:12.917558 2752 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/minicluster-data/ts-2/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/minicluster-data/ts-2/wal
uuid: "d06513b545e943768b9b5c7dccd5e209"
format_stamp: "Formatted at 2026-09-29 10:37:12 on dist-test-slave-vg89"
server_key: "64b002d3d9eeff2dfdbb4907797368cd"
server_key_iv: "6ad118e633f61d396c0c520f208c83d0"
server_key_version: "encryptionkey@0"
I20260929 10:37:12.917663 2752 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:37:12.934346 2752 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:37:12.937218 2752 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:37:12.937428 2752 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:37:12.938012 2752 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:37:12.938910 2752 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:37:12.938979 2752 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:12.939049 2752 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:37:12.939090 2752 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:12.948007 2752 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.67:38949
I20260929 10:37:12.948032 2881 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.67:38949 every 8 connection(s)
I20260929 10:37:12.948930 2752 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
I20260929 10:37:12.954082 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 2752
I20260929 10:37:12.954231 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/minicluster-data/ts-2/wal/instance
I20260929 10:37:12.954609 15689 external_mini_cluster.cc:1428] Setting key 4e9a28f9f3c4d507d791632d535942e7
Sep 29 10:37:12 dist-test-slave-vg89 krb5kdc[2377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678232, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.67@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:37:12.962383 2461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:12.950703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.67:50401 (local address 127.15.82.126:44079)
0929 10:37:12.950864 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:12.950868 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:12.951694 (+ 826us) server_negotiation.cc:407] Connection header received
0929 10:37:12.953043 (+ 1349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:12.953047 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:12.953103 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:12.953204 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:12.954504 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:12.955210 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:12.956494 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:12.956750 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:12.956844 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:12.959302 (+ 2458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:12.959305 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:12.959307 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:37:12.959329 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:12.960580 (+ 1251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:12.961076 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:12.961079 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:12.961080 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:12.961160 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:12.961465 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:12.961468 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:12.961469 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:12.961635 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:12.961713 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:12.962157 (+ 444us) server_negotiation.cc:299] Negotiation successful
0929 10:37:12.962255 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260929 10:37:12.963161 2884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:12.951113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:44079 (local address 127.15.82.67:50401)
0929 10:37:12.951528 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:12.951567 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 10:37:12.952832 (+ 1265us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:37:12.953388 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:37:12.953394 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:37:12.953723 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:37:12.954351 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:12.954366 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:12.955346 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:12.955349 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:12.956374 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:12.956383 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:12.956877 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:12.956881 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:12.957050 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:12.957623 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:37:12.957638 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:37:12.959152 (+ 1514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:37:12.960690 (+ 1538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:37:12.960695 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:37:12.960705 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:37:12.960963 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:37:12.961256 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:37:12.961258 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:37:12.961259 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:37:12.961382 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:37:12.961718 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:37:12.961723 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:37:12.961929 (+ 206us) client_negotiation.cc:769] Sending connection context
0929 10:37:12.962084 (+ 155us) client_negotiation.cc:240] Negotiation successful
0929 10:37:12.962326 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":119,"threads_started":1}
I20260929 10:37:12.964193 2882 heartbeater.cc:347] Connected to a master server at 127.15.82.126:44079
I20260929 10:37:12.964416 2882 heartbeater.cc:464] Registering TS with master...
I20260929 10:37:12.964912 2882 heartbeater.cc:511] Master 127.15.82.126:44079 requested a full tablet report, sending...
I20260929 10:37:12.965848 2415 ts_manager.cc:194] Registered new tserver with Master: d06513b545e943768b9b5c7dccd5e209 (127.15.82.67:38949)
I20260929 10:37:12.966526 2415 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.82.67@KRBTEST.COM'} at 127.15.82.67:50401
I20260929 10:37:12.968394 15689 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 10:37:12.978078 2461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:12.970390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47938 (local address 127.15.82.126:44079)
0929 10:37:12.970635 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:12.970640 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:12.970895 (+ 255us) server_negotiation.cc:407] Connection header received
0929 10:37:12.970986 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:12.970988 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:12.971027 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:12.971096 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:12.971760 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:12.972278 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:12.973190 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:12.973414 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:12.973507 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:12.974988 (+ 1481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:12.974991 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:12.974993 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:37:12.975015 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:12.976366 (+ 1351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:12.976839 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:12.976842 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:12.976843 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:12.976918 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:12.977166 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:12.977168 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:12.977170 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:12.977383 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:12.977537 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:12.977803 (+ 266us) server_negotiation.cc:299] Negotiation successful
0929 10:37:12.977902 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129}
I20260929 10:37:12.981096 2415 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47938:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 10:37:12.984265 2415 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:37:12.998796 2894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:12.992663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34962 (local address 127.15.82.65:41389)
0929 10:37:12.992948 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:12.992966 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:12.993152 (+ 186us) server_negotiation.cc:407] Connection header received
0929 10:37:12.993388 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:12.993398 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:12.993621 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:12.993820 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:37:12.994144 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:12.995342 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:12.997196 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:12.998062 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:12.998174 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:12.998322 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:12.998544 (+ 222us) server_negotiation.cc:299] Negotiation successful
0929 10:37:12.998637 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":108,"threads_started":1}
I20260929 10:37:12.998788 2893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:12.992531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.65:41389 (local address 127.0.0.1:34962)
0929 10:37:12.992998 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:12.993045 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0929 10:37:12.993198 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:37:12.993841 (+ 643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:37:12.993847 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:37:12.993873 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:37:12.994016 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:12.994023 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:12.995470 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:12.995473 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:12.997065 (+ 1592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:12.997075 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:12.998193 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:12.998199 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:12.998310 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:12.998348 (+ 38us) client_negotiation.cc:769] Sending connection context
0929 10:37:12.998510 (+ 162us) client_negotiation.cc:240] Negotiation successful
0929 10:37:12.998637 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":93,"threads_started":1}
I20260929 10:37:12.999468 2896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:12.993620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.66:32915 (local address 127.0.0.1:42434)
0929 10:37:12.994243 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:12.994256 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 10:37:12.994331 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:37:12.994996 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:37:12.994999 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:37:12.995015 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:37:12.995102 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:12.995107 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:12.996490 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:12.996493 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:12.997414 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:12.997424 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:12.998510 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:12.998516 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:12.998617 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:12.998641 (+ 24us) client_negotiation.cc:769] Sending connection context
0929 10:37:12.998706 (+ 65us) client_negotiation.cc:240] Negotiation successful
0929 10:37:12.998795 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"mutex_wait_us":37,"thread_start_us":145,"threads_started":1}
I20260929 10:37:12.999670 2897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:12.993623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52954 (local address 127.15.82.67:38949)
0929 10:37:12.994373 (+ 750us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:12.994384 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:12.994428 (+ 44us) server_negotiation.cc:407] Connection header received
0929 10:37:12.994500 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:12.994505 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:12.994684 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:12.994828 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:37:12.995103 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:12.996307 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:12.997620 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:12.998471 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:12.998875 (+ 404us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:12.999002 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:12.999049 (+ 47us) server_negotiation.cc:299] Negotiation successful
0929 10:37:12.999128 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":643,"thread_start_us":129,"threads_started":1}
I20260929 10:37:12.999698 2898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:12.994076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42434 (local address 127.15.82.66:32915)
0929 10:37:12.994515 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:12.994524 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:12.994540 (+ 16us) server_negotiation.cc:407] Connection header received
0929 10:37:12.994598 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:12.994601 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:12.994757 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:12.994928 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:37:12.995215 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:12.996323 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:12.997543 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:12.998142 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:12.998250 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:12.998425 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:12.998875 (+ 450us) server_negotiation.cc:299] Negotiation successful
0929 10:37:12.999136 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":101,"threads_started":1}
I20260929 10:37:12.999567 2895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:12.993265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.67:38949 (local address 127.0.0.1:52954)
0929 10:37:12.994010 (+ 745us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:12.994023 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 10:37:12.994132 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:37:12.994823 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:37:12.994828 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:37:12.994851 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:37:12.994956 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:12.994965 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:12.996451 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:12.996454 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:12.997501 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:12.997513 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:12.998595 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:12.998598 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:12.998682 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:12.998710 (+ 28us) client_negotiation.cc:769] Sending connection context
0929 10:37:12.998753 (+ 43us) client_negotiation.cc:240] Negotiation successful
0929 10:37:12.998804 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":682,"mutex_wait_us":31,"thread_start_us":121,"threads_started":1}
I20260929 10:37:13.002210 2547 tablet_service.cc:1467] Processing CreateTablet for tablet 61fb8836a5594976aa1e875d3490f108 (DEFAULT_TABLE table=test-table [id=fee2b133f2a746ce99b9259c4f04d6a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:37:13.002473 2816 tablet_service.cc:1467] Processing CreateTablet for tablet 61fb8836a5594976aa1e875d3490f108 (DEFAULT_TABLE table=test-table [id=fee2b133f2a746ce99b9259c4f04d6a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:37:13.002475 2681 tablet_service.cc:1467] Processing CreateTablet for tablet 61fb8836a5594976aa1e875d3490f108 (DEFAULT_TABLE table=test-table [id=fee2b133f2a746ce99b9259c4f04d6a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:37:13.003413 2547 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61fb8836a5594976aa1e875d3490f108. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:37:13.003443 2681 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61fb8836a5594976aa1e875d3490f108. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:37:13.003448 2816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61fb8836a5594976aa1e875d3490f108. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:37:13.009418 2899 tablet_bootstrap.cc:492] T 61fb8836a5594976aa1e875d3490f108 P 0df50643c6eb4749a0d587dbb8cf9132: Bootstrap starting.
I20260929 10:37:13.010855 2901 tablet_bootstrap.cc:492] T 61fb8836a5594976aa1e875d3490f108 P d06513b545e943768b9b5c7dccd5e209: Bootstrap starting.
I20260929 10:37:13.012059 2899 tablet_bootstrap.cc:654] T 61fb8836a5594976aa1e875d3490f108 P 0df50643c6eb4749a0d587dbb8cf9132: Neither blocks nor log segments found. Creating new log.
I20260929 10:37:13.012956 2901 tablet_bootstrap.cc:654] T 61fb8836a5594976aa1e875d3490f108 P d06513b545e943768b9b5c7dccd5e209: Neither blocks nor log segments found. Creating new log.
I20260929 10:37:13.013437 2899 log.cc:824] T 61fb8836a5594976aa1e875d3490f108 P 0df50643c6eb4749a0d587dbb8cf9132: Log is configured to *not* fsync() on all Append() calls
I20260929 10:37:13.014173 2900 tablet_bootstrap.cc:492] T 61fb8836a5594976aa1e875d3490f108 P ae2547afea1f4ae1a8aef33eed4421a4: Bootstrap starting.
I20260929 10:37:13.014206 2901 log.cc:824] T 61fb8836a5594976aa1e875d3490f108 P d06513b545e943768b9b5c7dccd5e209: Log is configured to *not* fsync() on all Append() calls
I20260929 10:37:13.015472 2899 tablet_bootstrap.cc:492] T 61fb8836a5594976aa1e875d3490f108 P 0df50643c6eb4749a0d587dbb8cf9132: No bootstrap required, opened a new log
I20260929 10:37:13.015679 2899 ts_tablet_manager.cc:1397] T 61fb8836a5594976aa1e875d3490f108 P 0df50643c6eb4749a0d587dbb8cf9132: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260929 10:37:13.016203 2901 tablet_bootstrap.cc:492] T 61fb8836a5594976aa1e875d3490f108 P d06513b545e943768b9b5c7dccd5e209: No bootstrap required, opened a new log
I20260929 10:37:13.016419 2901 ts_tablet_manager.cc:1397] T 61fb8836a5594976aa1e875d3490f108 P d06513b545e943768b9b5c7dccd5e209: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260929 10:37:13.016692 2900 tablet_bootstrap.cc:654] T 61fb8836a5594976aa1e875d3490f108 P ae2547afea1f4ae1a8aef33eed4421a4: Neither blocks nor log segments found. Creating new log.
I20260929 10:37:13.017537 2900 log.cc:824] T 61fb8836a5594976aa1e875d3490f108 P ae2547afea1f4ae1a8aef33eed4421a4: Log is configured to *not* fsync() on all Append() calls
I20260929 10:37:13.019238 2899 raft_consensus.cc:348] T 61fb8836a5594976aa1e875d3490f108 P 0df50643c6eb4749a0d587dbb8cf9132 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0df50643c6eb4749a0d587dbb8cf9132" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 41389 } } peers { permanent_uuid: "d06513b545e943768b9b5c7dccd5e209" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 38949 } } peers { permanent_uuid: "ae2547afea1f4ae1a8aef33eed4421a4" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 32915 } }
I20260929 10:37:13.019476 2899 raft_consensus.cc:374] T 61fb8836a5594976aa1e875d3490f108 P 0df50643c6eb4749a0d587dbb8cf9132 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:37:13.019498 2900 tablet_bootstrap.cc:492] T 61fb8836a5594976aa1e875d3490f108 P ae2547afea1f4ae1a8aef33eed4421a4: No bootstrap required, opened a new log
I20260929 10:37:13.019550 2899 raft_consensus.cc:729] T 61fb8836a5594976aa1e875d3490f108 P 0df50643c6eb4749a0d587dbb8cf9132 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0df50643c6eb4749a0d587dbb8cf9132, State: Initialized, Role: FOLLOWER
I20260929 10:37:13.019716 2900 ts_tablet_manager.cc:1397] T 61fb8836a5594976aa1e875d3490f108 P ae2547afea1f4ae1a8aef33eed4421a4: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260929 10:37:13.020008 2899 consensus_queue.cc:260] T 61fb8836a5594976aa1e875d3490f108 P 0df50643c6eb4749a0d587dbb8cf9132 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0df50643c6eb4749a0d587dbb8cf9132" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 41389 } } peers { permanent_uuid: "d06513b545e943768b9b5c7dccd5e209" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 38949 } } peers { permanent_uuid: "ae2547afea1f4ae1a8aef33eed4421a4" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 32915 } }
I20260929 10:37:13.020264 2901 raft_consensus.cc:348] T 61fb8836a5594976aa1e875d3490f108 P d06513b545e943768b9b5c7dccd5e209 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0df50643c6eb4749a0d587dbb8cf9132" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 41389 } } peers { permanent_uuid: "d06513b545e943768b9b5c7dccd5e209" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 38949 } } peers { permanent_uuid: "ae2547afea1f4ae1a8aef33eed4421a4" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 32915 } }
I20260929 10:37:13.020685 2613 heartbeater.cc:503] Master 127.15.82.126:44079 was elected leader, sending a full tablet report...
I20260929 10:37:13.021096 2899 ts_tablet_manager.cc:1428] T 61fb8836a5594976aa1e875d3490f108 P 0df50643c6eb4749a0d587dbb8cf9132: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260929 10:37:13.021656 2901 raft_consensus.cc:374] T 61fb8836a5594976aa1e875d3490f108 P d06513b545e943768b9b5c7dccd5e209 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:37:13.021731 2901 raft_consensus.cc:729] T 61fb8836a5594976aa1e875d3490f108 P d06513b545e943768b9b5c7dccd5e209 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d06513b545e943768b9b5c7dccd5e209, State: Initialized, Role: FOLLOWER
I20260929 10:37:13.022249 2901 consensus_queue.cc:260] T 61fb8836a5594976aa1e875d3490f108 P d06513b545e943768b9b5c7dccd5e209 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0df50643c6eb4749a0d587dbb8cf9132" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 41389 } } peers { permanent_uuid: "d06513b545e943768b9b5c7dccd5e209" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 38949 } } peers { permanent_uuid: "ae2547afea1f4ae1a8aef33eed4421a4" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 32915 } }
I20260929 10:37:13.022899 2882 heartbeater.cc:503] Master 127.15.82.126:44079 was elected leader, sending a full tablet report...
I20260929 10:37:13.023196 2901 ts_tablet_manager.cc:1428] T 61fb8836a5594976aa1e875d3490f108 P d06513b545e943768b9b5c7dccd5e209: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260929 10:37:13.023876 2900 raft_consensus.cc:348] T 61fb8836a5594976aa1e875d3490f108 P ae2547afea1f4ae1a8aef33eed4421a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0df50643c6eb4749a0d587dbb8cf9132" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 41389 } } peers { permanent_uuid: "d06513b545e943768b9b5c7dccd5e209" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 38949 } } peers { permanent_uuid: "ae2547afea1f4ae1a8aef33eed4421a4" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 32915 } }
I20260929 10:37:13.024119 2900 raft_consensus.cc:374] T 61fb8836a5594976aa1e875d3490f108 P ae2547afea1f4ae1a8aef33eed4421a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:37:13.024169 2900 raft_consensus.cc:729] T 61fb8836a5594976aa1e875d3490f108 P ae2547afea1f4ae1a8aef33eed4421a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae2547afea1f4ae1a8aef33eed4421a4, State: Initialized, Role: FOLLOWER
I20260929 10:37:13.024561 2900 consensus_queue.cc:260] T 61fb8836a5594976aa1e875d3490f108 P ae2547afea1f4ae1a8aef33eed4421a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0df50643c6eb4749a0d587dbb8cf9132" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 41389 } } peers { permanent_uuid: "d06513b545e943768b9b5c7dccd5e209" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 38949 } } peers { permanent_uuid: "ae2547afea1f4ae1a8aef33eed4421a4" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 32915 } }
I20260929 10:37:13.025197 2747 heartbeater.cc:503] Master 127.15.82.126:44079 was elected leader, sending a full tablet report...
I20260929 10:37:13.026346 2900 ts_tablet_manager.cc:1428] T 61fb8836a5594976aa1e875d3490f108 P ae2547afea1f4ae1a8aef33eed4421a4: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260929 10:37:13.066325 2905 raft_consensus.cc:482] T 61fb8836a5594976aa1e875d3490f108 P 0df50643c6eb4749a0d587dbb8cf9132 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:37:13.066617 2905 raft_consensus.cc:504] T 61fb8836a5594976aa1e875d3490f108 P 0df50643c6eb4749a0d587dbb8cf9132 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0df50643c6eb4749a0d587dbb8cf9132" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 41389 } } peers { permanent_uuid: "d06513b545e943768b9b5c7dccd5e209" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 38949 } } peers { permanent_uuid: "ae2547afea1f4ae1a8aef33eed4421a4" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 32915 } }
I20260929 10:37:13.067757 2905 leader_election.cc:288] T 61fb8836a5594976aa1e875d3490f108 P 0df50643c6eb4749a0d587dbb8cf9132 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d06513b545e943768b9b5c7dccd5e209 (127.15.82.67:38949), ae2547afea1f4ae1a8aef33eed4421a4 (127.15.82.66:32915)
I20260929 10:37:13.072472 2897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:13.068142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.65:45433 (local address 127.15.82.67:38949)
0929 10:37:13.068299 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:13.068306 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:13.068434 (+ 128us) server_negotiation.cc:407] Connection header received
0929 10:37:13.068661 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:13.068666 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:13.068719 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:13.068802 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:37:13.069152 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:13.069796 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:13.071286 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:13.071903 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:13.072040 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:13.072096 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:13.072278 (+ 182us) server_negotiation.cc:299] Negotiation successful
0929 10:37:13.072350 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260929 10:37:13.072486 2908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:13.068042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.67:38949 (local address 127.15.82.65:45433)
0929 10:37:13.068343 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:13.068356 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 10:37:13.068487 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:37:13.068819 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:37:13.068822 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:37:13.068868 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:37:13.069004 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:13.069026 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:13.069912 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:13.069914 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:13.071163 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:13.071173 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:13.072029 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:13.072033 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:13.072138 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:13.072161 (+ 23us) client_negotiation.cc:769] Sending connection context
0929 10:37:13.072263 (+ 102us) client_negotiation.cc:240] Negotiation successful
0929 10:37:13.072336 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"mutex_wait_us":26,"thread_start_us":123,"threads_started":1}
I20260929 10:37:13.072602 2898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:13.068649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.65:56761 (local address 127.15.82.66:32915)
0929 10:37:13.068817 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:13.068821 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:13.069044 (+ 223us) server_negotiation.cc:407] Connection header received
0929 10:37:13.069152 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:13.069155 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:13.069202 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:13.069270 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:37:13.069532 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:13.070154 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:13.071486 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:13.072029 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:13.072156 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:13.072209 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:13.072410 (+ 201us) server_negotiation.cc:299] Negotiation successful
0929 10:37:13.072481 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260929 10:37:13.072642 2909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:13.068487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.66:32915 (local address 127.15.82.65:56761)
0929 10:37:13.068955 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:13.068974 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 10:37:13.069048 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:37:13.069302 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:37:13.069305 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:37:13.069321 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:37:13.069428 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:13.069434 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:13.070263 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:13.070266 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:13.071376 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:13.071383 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:13.072147 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:13.072151 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:13.072279 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:13.072305 (+ 26us) client_negotiation.cc:769] Sending connection context
0929 10:37:13.072394 (+ 89us) client_negotiation.cc:240] Negotiation successful
0929 10:37:13.072463 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":400,"mutex_wait_us":1,"thread_start_us":104,"threads_started":1}
I20260929 10:37:13.073267 2836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61fb8836a5594976aa1e875d3490f108" candidate_uuid: "0df50643c6eb4749a0d587dbb8cf9132" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d06513b545e943768b9b5c7dccd5e209" is_pre_election: true
I20260929 10:37:13.073263 2701 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61fb8836a5594976aa1e875d3490f108" candidate_uuid: "0df50643c6eb4749a0d587dbb8cf9132" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae2547afea1f4ae1a8aef33eed4421a4" is_pre_election: true
I20260929 10:37:13.073613 2836 raft_consensus.cc:2454] T 61fb8836a5594976aa1e875d3490f108 P d06513b545e943768b9b5c7dccd5e209 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0df50643c6eb4749a0d587dbb8cf9132 in term 0.
I20260929 10:37:13.073613 2701 raft_consensus.cc:2454] T 61fb8836a5594976aa1e875d3490f108 P ae2547afea1f4ae1a8aef33eed4421a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0df50643c6eb4749a0d587dbb8cf9132 in term 0.
I20260929 10:37:13.074085 2503 leader_election.cc:302] T 61fb8836a5594976aa1e875d3490f108 P 0df50643c6eb4749a0d587dbb8cf9132 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0df50643c6eb4749a0d587dbb8cf9132, ae2547afea1f4ae1a8aef33eed4421a4; no voters:
I20260929 10:37:13.074337 2905 raft_consensus.cc:2781] T 61fb8836a5594976aa1e875d3490f108 P 0df50643c6eb4749a0d587dbb8cf9132 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:37:13.074466 2905 raft_consensus.cc:482] T 61fb8836a5594976aa1e875d3490f108 P 0df50643c6eb4749a0d587dbb8cf9132 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:37:13.074530 2905 raft_consensus.cc:3037] T 61fb8836a5594976aa1e875d3490f108 P 0df50643c6eb4749a0d587dbb8cf9132 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:37:13.075579 2905 raft_consensus.cc:504] T 61fb8836a5594976aa1e875d3490f108 P 0df50643c6eb4749a0d587dbb8cf9132 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0df50643c6eb4749a0d587dbb8cf9132" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 41389 } } peers { permanent_uuid: "d06513b545e943768b9b5c7dccd5e209" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 38949 } } peers { permanent_uuid: "ae2547afea1f4ae1a8aef33eed4421a4" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 32915 } }
I20260929 10:37:13.075938 2905 leader_election.cc:288] T 61fb8836a5594976aa1e875d3490f108 P 0df50643c6eb4749a0d587dbb8cf9132 [CANDIDATE]: Term 1 election: Requested vote from peers d06513b545e943768b9b5c7dccd5e209 (127.15.82.67:38949), ae2547afea1f4ae1a8aef33eed4421a4 (127.15.82.66:32915)
I20260929 10:37:13.076285 2836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61fb8836a5594976aa1e875d3490f108" candidate_uuid: "0df50643c6eb4749a0d587dbb8cf9132" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d06513b545e943768b9b5c7dccd5e209"
I20260929 10:37:13.076362 2701 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61fb8836a5594976aa1e875d3490f108" candidate_uuid: "0df50643c6eb4749a0d587dbb8cf9132" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae2547afea1f4ae1a8aef33eed4421a4"
I20260929 10:37:13.076426 2836 raft_consensus.cc:3037] T 61fb8836a5594976aa1e875d3490f108 P d06513b545e943768b9b5c7dccd5e209 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:37:13.076535 2701 raft_consensus.cc:3037] T 61fb8836a5594976aa1e875d3490f108 P ae2547afea1f4ae1a8aef33eed4421a4 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:37:13.077492 2836 raft_consensus.cc:2454] T 61fb8836a5594976aa1e875d3490f108 P d06513b545e943768b9b5c7dccd5e209 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0df50643c6eb4749a0d587dbb8cf9132 in term 1.
I20260929 10:37:13.077579 2701 raft_consensus.cc:2454] T 61fb8836a5594976aa1e875d3490f108 P ae2547afea1f4ae1a8aef33eed4421a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0df50643c6eb4749a0d587dbb8cf9132 in term 1.
I20260929 10:37:13.077831 2503 leader_election.cc:302] T 61fb8836a5594976aa1e875d3490f108 P 0df50643c6eb4749a0d587dbb8cf9132 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0df50643c6eb4749a0d587dbb8cf9132, d06513b545e943768b9b5c7dccd5e209; no voters:
I20260929 10:37:13.078008 2905 raft_consensus.cc:2781] T 61fb8836a5594976aa1e875d3490f108 P 0df50643c6eb4749a0d587dbb8cf9132 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:37:13.078226 2905 raft_consensus.cc:686] T 61fb8836a5594976aa1e875d3490f108 P 0df50643c6eb4749a0d587dbb8cf9132 [term 1 LEADER]: Becoming Leader. State: Replica: 0df50643c6eb4749a0d587dbb8cf9132, State: Running, Role: LEADER
I20260929 10:37:13.078584 2905 consensus_queue.cc:237] T 61fb8836a5594976aa1e875d3490f108 P 0df50643c6eb4749a0d587dbb8cf9132 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0df50643c6eb4749a0d587dbb8cf9132" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 41389 } } peers { permanent_uuid: "d06513b545e943768b9b5c7dccd5e209" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 38949 } } peers { permanent_uuid: "ae2547afea1f4ae1a8aef33eed4421a4" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 32915 } }
I20260929 10:37:13.081784 2415 catalog_manager.cc:5495] T 61fb8836a5594976aa1e875d3490f108 P 0df50643c6eb4749a0d587dbb8cf9132 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0df50643c6eb4749a0d587dbb8cf9132 (127.15.82.65). New cstate: current_term: 1 leader_uuid: "0df50643c6eb4749a0d587dbb8cf9132" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0df50643c6eb4749a0d587dbb8cf9132" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d06513b545e943768b9b5c7dccd5e209" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 38949 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ae2547afea1f4ae1a8aef33eed4421a4" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 32915 } health_report { overall_health: UNKNOWN } } }
I20260929 10:37:13.099061 2894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:13.095129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34970 (local address 127.15.82.65:41389)
0929 10:37:13.095265 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:13.095270 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:13.095284 (+ 14us) server_negotiation.cc:407] Connection header received
0929 10:37:13.095517 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:13.095519 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:13.095569 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:13.095676 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:37:13.095928 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:13.096439 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:13.097573 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:13.097805 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:13.097915 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:13.098213 (+ 298us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:37:13.098684 (+ 471us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:37:13.098822 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:13.098859 (+ 37us) server_negotiation.cc:299] Negotiation successful
0929 10:37:13.098915 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260929 10:37:13.104862 2701 raft_consensus.cc:1261] T 61fb8836a5594976aa1e875d3490f108 P ae2547afea1f4ae1a8aef33eed4421a4 [term 1 FOLLOWER]: Refusing update from remote peer 0df50643c6eb4749a0d587dbb8cf9132: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:37:13.104914 2836 raft_consensus.cc:1261] T 61fb8836a5594976aa1e875d3490f108 P d06513b545e943768b9b5c7dccd5e209 [term 1 FOLLOWER]: Refusing update from remote peer 0df50643c6eb4749a0d587dbb8cf9132: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:37:13.105544 2905 consensus_queue.cc:1035] T 61fb8836a5594976aa1e875d3490f108 P 0df50643c6eb4749a0d587dbb8cf9132 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae2547afea1f4ae1a8aef33eed4421a4" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 32915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:37:13.105793 2910 consensus_queue.cc:1035] T 61fb8836a5594976aa1e875d3490f108 P 0df50643c6eb4749a0d587dbb8cf9132 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d06513b545e943768b9b5c7dccd5e209" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 38949 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:37:13.121685 2417 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47938:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:37:13.121831 2417 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47938:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:37:13.124284 2417 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a7efcd953c6d4fafbb2a42b73eb2e950: Sending DeleteTablet for 3 replicas of tablet 61fb8836a5594976aa1e875d3490f108
I20260929 10:37:13.124976 2547 tablet_service.cc:1514] Processing DeleteTablet for tablet 61fb8836a5594976aa1e875d3490f108 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:37:13 UTC) from {username='kudu', principal='kudu/127.15.82.126@KRBTEST.COM'} at 127.0.0.1:34962
I20260929 10:37:13.125046 2816 tablet_service.cc:1514] Processing DeleteTablet for tablet 61fb8836a5594976aa1e875d3490f108 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:37:13 UTC) from {username='kudu', principal='kudu/127.15.82.126@KRBTEST.COM'} at 127.0.0.1:52954
I20260929 10:37:13.125434 2922 tablet_replica.cc:330] T 61fb8836a5594976aa1e875d3490f108 P 0df50643c6eb4749a0d587dbb8cf9132: stopping tablet replica
I20260929 10:37:13.125469 2681 tablet_service.cc:1514] Processing DeleteTablet for tablet 61fb8836a5594976aa1e875d3490f108 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:37:13 UTC) from {username='kudu', principal='kudu/127.15.82.126@KRBTEST.COM'} at 127.0.0.1:42434
I20260929 10:37:13.125763 2922 raft_consensus.cc:2229] T 61fb8836a5594976aa1e875d3490f108 P 0df50643c6eb4749a0d587dbb8cf9132 [term 1 LEADER]: Raft consensus shutting down.
I20260929 10:37:13.126164 2922 raft_consensus.cc:2258] T 61fb8836a5594976aa1e875d3490f108 P 0df50643c6eb4749a0d587dbb8cf9132 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:37:13.126183 2923 tablet_replica.cc:330] T 61fb8836a5594976aa1e875d3490f108 P d06513b545e943768b9b5c7dccd5e209: stopping tablet replica
I20260929 10:37:13.126602 2923 raft_consensus.cc:2229] T 61fb8836a5594976aa1e875d3490f108 P d06513b545e943768b9b5c7dccd5e209 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:37:13.126637 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 2484
I20260929 10:37:13.126901 2923 raft_consensus.cc:2258] T 61fb8836a5594976aa1e875d3490f108 P d06513b545e943768b9b5c7dccd5e209 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:37:13.128455 2923 ts_tablet_manager.cc:1905] T 61fb8836a5594976aa1e875d3490f108 P d06513b545e943768b9b5c7dccd5e209: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:37:13.125991 2924 tablet_replica.cc:330] T 61fb8836a5594976aa1e875d3490f108 P ae2547afea1f4ae1a8aef33eed4421a4: stopping tablet replica
I20260929 10:37:13.130813 2923 ts_tablet_manager.cc:1918] T 61fb8836a5594976aa1e875d3490f108 P d06513b545e943768b9b5c7dccd5e209: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:37:13.130842 2924 raft_consensus.cc:2229] T 61fb8836a5594976aa1e875d3490f108 P ae2547afea1f4ae1a8aef33eed4421a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:37:13.130921 2923 log.cc:1196] T 61fb8836a5594976aa1e875d3490f108 P d06513b545e943768b9b5c7dccd5e209: Deleting WAL directory at /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/minicluster-data/ts-2/wal/wals/61fb8836a5594976aa1e875d3490f108
I20260929 10:37:13.131137 2924 raft_consensus.cc:2258] T 61fb8836a5594976aa1e875d3490f108 P ae2547afea1f4ae1a8aef33eed4421a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:37:13.131264 2923 ts_tablet_manager.cc:1939] T 61fb8836a5594976aa1e875d3490f108 P d06513b545e943768b9b5c7dccd5e209: Deleting consensus metadata
I20260929 10:37:13.132329 2924 ts_tablet_manager.cc:1905] T 61fb8836a5594976aa1e875d3490f108 P ae2547afea1f4ae1a8aef33eed4421a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:37:13.132586 2409 catalog_manager.cc:4841] TS d06513b545e943768b9b5c7dccd5e209 (127.15.82.67:38949): tablet 61fb8836a5594976aa1e875d3490f108 (table test-table [id=fee2b133f2a746ce99b9259c4f04d6a0]) successfully deleted
W20260929 10:37:13.134179 2409 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.82.65:41389 (error 108)
I20260929 10:37:13.134631 2924 ts_tablet_manager.cc:1918] T 61fb8836a5594976aa1e875d3490f108 P ae2547afea1f4ae1a8aef33eed4421a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:37:13.134743 2924 log.cc:1196] T 61fb8836a5594976aa1e875d3490f108 P ae2547afea1f4ae1a8aef33eed4421a4: Deleting WAL directory at /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790677981592896-15689-0/minicluster-data/ts-1/wal/wals/61fb8836a5594976aa1e875d3490f108
I20260929 10:37:13.135102 2924 ts_tablet_manager.cc:1939] T 61fb8836a5594976aa1e875d3490f108 P ae2547afea1f4ae1a8aef33eed4421a4: Deleting consensus metadata
I20260929 10:37:13.135264 2896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:13.134926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.65:41389 (local address 127.0.0.1:34974)
0929 10:37:13.135090 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:13.135168 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.65:41389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260929 10:37:13.135285 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 2618
W20260929 10:37:13.135946 2409 catalog_manager.cc:4584] TS 0df50643c6eb4749a0d587dbb8cf9132 (127.15.82.65:41389): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 61fb8836a5594976aa1e875d3490f108: Network error: Client connection negotiation failed: client connection to 127.15.82.65:41389: connect: Connection refused (error 111)
I20260929 10:37:13.140460 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 2752
I20260929 10:37:13.141266 2896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:13.140620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.66:32915 (local address 127.0.0.1:42448)
0929 10:37:13.141139 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:13.141192 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.66:32915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":448}
I20260929 10:37:13.145694 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 2390
2026-09-29T10:37:13Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2284 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-29T10:37:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:37:13Z Disabled control of system clock
I20260929 10:37:13.176558 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790677981592896-15689-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790677981592896-15689-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790677981592896-15689-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:36025
--webserver_interface=127.15.82.126
--webserver_port=0
--builtin_ntp_servers=127.15.82.84:43357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.82.126:36025
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 10:37:13.288651 2929 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:37:13.288925 2929 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:37:13.288982 2929 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:37:13.293108 2929 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:37:13.293164 2929 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:37:13.293180 2929 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:37:13.293196 2929 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:37:13.293243 2929 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:37:13.297937 2929 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:43357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790677981592896-15689-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790677981592896-15689-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.82.126:36025
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:36025
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.82.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2929
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790677981592896-15689-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:37:13.299036 2929 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:37:13.300485 2929 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:37:13.309085 2934 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:13.309116 2937 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:13.309196 2935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:37:13.309501 2929 server_base.cc:1034] running on GCE node
I20260929 10:37:13.310040 2929 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:37:13.311196 2929 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:37:13.312378 2929 hybrid_clock.cc:648] HybridClock initialized: now 1790678233312351 us; error 65 us; skew 500 ppm
I20260929 10:37:13.314136 2929 webserver.cc:460] Webserver started at http://127.15.82.126:34701/ using document root <none> and password file <none>
I20260929 10:37:13.314774 2929 fs_manager.cc:359] Metadata directory not provided
I20260929 10:37:13.314867 2929 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:37:13.315096 2929 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:37:13.317124 2929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790677981592896-15689-0/minicluster-data/master-0/data/instance:
uuid: "57c9a8cb43584e139991cb78a786b6e2"
format_stamp: "Formatted at 2026-09-29 10:37:13 on dist-test-slave-vg89"
server_key: "b42e98636fc637432a480fc2b45774c4"
server_key_iv: "22993c7a448c88d50c628a0cc8feb582"
server_key_version: "encryptionkey@0"
I20260929 10:37:13.317690 2929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790677981592896-15689-0/minicluster-data/master-0/wal/instance:
uuid: "57c9a8cb43584e139991cb78a786b6e2"
format_stamp: "Formatted at 2026-09-29 10:37:13 on dist-test-slave-vg89"
server_key: "b42e98636fc637432a480fc2b45774c4"
server_key_iv: "22993c7a448c88d50c628a0cc8feb582"
server_key_version: "encryptionkey@0"
I20260929 10:37:13.321345 2929 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260929 10:37:13.323911 2943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:13.325148 2929 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260929 10:37:13.325282 2929 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790677981592896-15689-0/minicluster-data/master-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790677981592896-15689-0/minicluster-data/master-0/wal
uuid: "57c9a8cb43584e139991cb78a786b6e2"
format_stamp: "Formatted at 2026-09-29 10:37:13 on dist-test-slave-vg89"
server_key: "b42e98636fc637432a480fc2b45774c4"
server_key_iv: "22993c7a448c88d50c628a0cc8feb582"
server_key_version: "encryptionkey@0"
I20260929 10:37:13.325402 2929 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790677981592896-15689-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790677981592896-15689-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790677981592896-15689-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:37:13.340349 2929 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:37:13.340989 2929 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:37:13.341179 2929 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:37:13.348192 2929 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.126:36025
I20260929 10:37:13.348198 2995 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.126:36025 every 8 connection(s)
I20260929 10:37:13.349094 2929 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
I20260929 10:37:13.352270 2996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:37:13.352420 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 2929
I20260929 10:37:13.352555 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790677981592896-15689-0/minicluster-data/master-0/wal/instance
I20260929 10:37:13.352837 15689 external_mini_cluster.cc:1428] Setting key 9e04b24945ec1d69006225e89e7d5eee
I20260929 10:37:13.359272 2996 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57c9a8cb43584e139991cb78a786b6e2: Bootstrap starting.
I20260929 10:37:13.361805 2996 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 57c9a8cb43584e139991cb78a786b6e2: Neither blocks nor log segments found. Creating new log.
I20260929 10:37:13.362249 2999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:13.354154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38014 (local address 127.15.82.126:36025)
0929 10:37:13.354777 (+ 623us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:13.354792 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:13.354828 (+ 36us) server_negotiation.cc:407] Connection header received
0929 10:37:13.355535 (+ 707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:13.355552 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:13.355932 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:13.356204 (+ 272us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:13.356541 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:13.357469 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:13.358792 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:13.359563 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:13.359681 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:13.360575 (+ 894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:13.360580 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:13.360591 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:37:13.360960 (+ 369us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:13.361014 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:37:13.361033 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:13.361107 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:13.361266 (+ 159us) server_negotiation.cc:299] Negotiation successful
0929 10:37:13.361553 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":143,"threads_started":1}
I20260929 10:37:13.362603 2996 log.cc:824] T 00000000000000000000000000000000 P 57c9a8cb43584e139991cb78a786b6e2: Log is configured to *not* fsync() on all Append() calls
I20260929 10:37:13.364393 2996 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57c9a8cb43584e139991cb78a786b6e2: No bootstrap required, opened a new log
I20260929 10:37:13.368000 2996 raft_consensus.cc:348] T 00000000000000000000000000000000 P 57c9a8cb43584e139991cb78a786b6e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57c9a8cb43584e139991cb78a786b6e2" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 36025 } }
I20260929 10:37:13.368196 2996 raft_consensus.cc:374] T 00000000000000000000000000000000 P 57c9a8cb43584e139991cb78a786b6e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:37:13.368268 2996 raft_consensus.cc:729] T 00000000000000000000000000000000 P 57c9a8cb43584e139991cb78a786b6e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57c9a8cb43584e139991cb78a786b6e2, State: Initialized, Role: FOLLOWER
I20260929 10:37:13.368708 2996 consensus_queue.cc:260] T 00000000000000000000000000000000 P 57c9a8cb43584e139991cb78a786b6e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57c9a8cb43584e139991cb78a786b6e2" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 36025 } }
I20260929 10:37:13.368857 2996 raft_consensus.cc:388] T 00000000000000000000000000000000 P 57c9a8cb43584e139991cb78a786b6e2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:37:13.368934 2996 raft_consensus.cc:482] T 00000000000000000000000000000000 P 57c9a8cb43584e139991cb78a786b6e2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:37:13.369045 2996 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 57c9a8cb43584e139991cb78a786b6e2 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:37:13.369879 2996 raft_consensus.cc:504] T 00000000000000000000000000000000 P 57c9a8cb43584e139991cb78a786b6e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57c9a8cb43584e139991cb78a786b6e2" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 36025 } }
I20260929 10:37:13.370203 2996 leader_election.cc:302] T 00000000000000000000000000000000 P 57c9a8cb43584e139991cb78a786b6e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 57c9a8cb43584e139991cb78a786b6e2; no voters:
I20260929 10:37:13.370530 2996 leader_election.cc:288] T 00000000000000000000000000000000 P 57c9a8cb43584e139991cb78a786b6e2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:37:13.370639 3001 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 57c9a8cb43584e139991cb78a786b6e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:37:13.370883 3001 raft_consensus.cc:686] T 00000000000000000000000000000000 P 57c9a8cb43584e139991cb78a786b6e2 [term 1 LEADER]: Becoming Leader. State: Replica: 57c9a8cb43584e139991cb78a786b6e2, State: Running, Role: LEADER
I20260929 10:37:13.371176 3001 consensus_queue.cc:237] T 00000000000000000000000000000000 P 57c9a8cb43584e139991cb78a786b6e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57c9a8cb43584e139991cb78a786b6e2" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 36025 } }
I20260929 10:37:13.371724 2996 sys_catalog.cc:564] T 00000000000000000000000000000000 P 57c9a8cb43584e139991cb78a786b6e2 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:37:13.372819 3003 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57c9a8cb43584e139991cb78a786b6e2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57c9a8cb43584e139991cb78a786b6e2. Latest consensus state: current_term: 1 leader_uuid: "57c9a8cb43584e139991cb78a786b6e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57c9a8cb43584e139991cb78a786b6e2" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 36025 } } }
I20260929 10:37:13.372942 3003 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57c9a8cb43584e139991cb78a786b6e2 [sys.catalog]: This master's current role is: LEADER
I20260929 10:37:13.373210 3002 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57c9a8cb43584e139991cb78a786b6e2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "57c9a8cb43584e139991cb78a786b6e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57c9a8cb43584e139991cb78a786b6e2" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 36025 } } }
I20260929 10:37:13.373315 3002 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57c9a8cb43584e139991cb78a786b6e2 [sys.catalog]: This master's current role is: LEADER
I20260929 10:37:13.373565 3010 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:37:13.376106 3010 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:37:13.380985 3010 catalog_manager.cc:1347] Generated new cluster ID: 4fcdb05f13e24a8fa50f2a028ac5c3b9
I20260929 10:37:13.381048 3010 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:37:13.393649 3010 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:37:13.394570 3010 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:37:13.408232 3010 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 57c9a8cb43584e139991cb78a786b6e2: Generated new TSK 0
I20260929 10:37:13.409153 3010 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 10:37:13.420025 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790677981592896-15689-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790677981592896-15689-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790677981592896-15689-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.65:0
--local_ip_for_outbound_sockets=127.15.82.65
--webserver_interface=127.15.82.65
--webserver_port=0
--tserver_master_addrs=127.15.82.126:36025
--builtin_ntp_servers=127.15.82.84:43357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 10:37:13.530316 3020 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:37:13.530656 3020 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:37:13.530725 3020 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:37:13.534678 3020 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:37:13.534758 3020 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:37:13.534852 3020 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.65
I20260929 10:37:13.539675 3020 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:43357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790677981592896-15689-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790677981592896-15689-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.82.65
--webserver_port=0
--tserver_master_addrs=127.15.82.126:36025
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3020
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.65
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790677981592896-15689-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:37:13.540761 3020 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:37:13.542155 3020 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:37:13.551779 3028 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:13.551806 3025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:13.551872 3026 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:37:13.552022 3020 server_base.cc:1034] running on GCE node
I20260929 10:37:13.552474 3020 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:37:13.553170 3020 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:37:13.554348 3020 hybrid_clock.cc:648] HybridClock initialized: now 1790678233554321 us; error 38 us; skew 500 ppm
I20260929 10:37:13.556262 3020 webserver.cc:460] Webserver started at http://127.15.82.65:46255/ using document root <none> and password file <none>
I20260929 10:37:13.556841 3020 fs_manager.cc:359] Metadata directory not provided
I20260929 10:37:13.556903 3020 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:37:13.557109 3020 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:37:13.559201 3020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790677981592896-15689-0/minicluster-data/ts-0/data/instance:
uuid: "08bfd5769d6f49f89be30c103e81b9af"
format_stamp: "Formatted at 2026-09-29 10:37:13 on dist-test-slave-vg89"
server_key: "84cc15b8b343f9560f1596104ef405a9"
server_key_iv: "7292c80dc2a1ec2c4eccf22b8152aba8"
server_key_version: "encryptionkey@0"
I20260929 10:37:13.559775 3020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790677981592896-15689-0/minicluster-data/ts-0/wal/instance:
uuid: "08bfd5769d6f49f89be30c103e81b9af"
format_stamp: "Formatted at 2026-09-29 10:37:13 on dist-test-slave-vg89"
server_key: "84cc15b8b343f9560f1596104ef405a9"
server_key_iv: "7292c80dc2a1ec2c4eccf22b8152aba8"
server_key_version: "encryptionkey@0"
I20260929 10:37:13.563750 3020 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260929 10:37:13.567204 3034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:13.568753 3020 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 10:37:13.568887 3020 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790677981592896-15689-0/minicluster-data/ts-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790677981592896-15689-0/minicluster-data/ts-0/wal
uuid: "08bfd5769d6f49f89be30c103e81b9af"
format_stamp: "Formatted at 2026-09-29 10:37:13 on dist-test-slave-vg89"
server_key: "84cc15b8b343f9560f1596104ef405a9"
server_key_iv: "7292c80dc2a1ec2c4eccf22b8152aba8"
server_key_version: "encryptionkey@0"
I20260929 10:37:13.569028 3020 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790677981592896-15689-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790677981592896-15689-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790677981592896-15689-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:37:13.580657 3020 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:37:13.581319 3020 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:37:13.581506 3020 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:37:13.582072 3020 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:37:13.583038 3020 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:37:13.583109 3020 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:13.583179 3020 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:37:13.583232 3020 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:13.592139 3020 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.65:33451
I20260929 10:37:13.592166 3147 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.65:33451 every 8 connection(s)
I20260929 10:37:13.593035 3020 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790677981592896-15689-0/minicluster-data/ts-0/data/info.pb
I20260929 10:37:13.596910 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 3020
I20260929 10:37:13.597082 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790677981592896-15689-0/minicluster-data/ts-0/wal/instance
I20260929 10:37:13.597493 15689 external_mini_cluster.cc:1428] Setting key aee63f929969d37c253fbc3a64de2f83
I20260929 10:37:13.599803 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790677981592896-15689-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790677981592896-15689-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790677981592896-15689-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.66:0
--local_ip_for_outbound_sockets=127.15.82.66
--webserver_interface=127.15.82.66
--webserver_port=0
--tserver_master_addrs=127.15.82.126:36025
--builtin_ntp_servers=127.15.82.84:43357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260929 10:37:13.603772 2999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:13.594945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.65:40325 (local address 127.15.82.126:36025)
0929 10:37:13.595087 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:13.595091 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:13.596120 (+ 1029us) server_negotiation.cc:407] Connection header received
0929 10:37:13.598055 (+ 1935us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:13.598059 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:13.598126 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:13.598221 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:13.599320 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:13.600130 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:13.601581 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:13.601830 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:13.601923 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:13.603296 (+ 1373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:13.603299 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:13.603302 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:37:13.603315 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:13.603334 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:37:13.603344 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:13.603408 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:13.603596 (+ 188us) server_negotiation.cc:299] Negotiation successful
0929 10:37:13.603652 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260929 10:37:13.604990 3150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:13.595256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:36025 (local address 127.15.82.65:40325)
0929 10:37:13.595963 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:13.596006 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 10:37:13.597739 (+ 1733us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:37:13.598468 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:37:13.598482 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:37:13.598575 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:37:13.599165 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:13.599186 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:13.600305 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:13.600310 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:13.601422 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:13.601437 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:13.601994 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:13.602000 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:13.602223 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:13.602973 (+ 750us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 10:37:13.602977 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:37:13.603034 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 10:37:13.603120 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:37:13.603432 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:37:13.603444 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:37:13.603454 (+ 10us) client_negotiation.cc:769] Sending connection context
0929 10:37:13.603584 (+ 130us) client_negotiation.cc:240] Negotiation successful
0929 10:37:13.603903 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":107,"threads_started":1}
I20260929 10:37:13.606316 3148 heartbeater.cc:347] Connected to a master server at 127.15.82.126:36025
I20260929 10:37:13.606585 3148 heartbeater.cc:464] Registering TS with master...
I20260929 10:37:13.607131 3148 heartbeater.cc:511] Master 127.15.82.126:36025 requested a full tablet report, sending...
I20260929 10:37:13.608726 2960 ts_manager.cc:194] Registered new tserver with Master: 08bfd5769d6f49f89be30c103e81b9af (127.15.82.65:33451)
I20260929 10:37:13.609968 2960 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.82.65:40325
W20260929 10:37:13.711903 3151 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:37:13.712164 3151 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:37:13.712203 3151 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:37:13.716262 3151 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:37:13.716327 3151 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:37:13.716383 3151 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.66
I20260929 10:37:13.721161 3151 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:43357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790677981592896-15689-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790677981592896-15689-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.82.66
--webserver_port=0
--tserver_master_addrs=127.15.82.126:36025
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3151
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.66
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790677981592896-15689-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:37:13.722273 3151 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:37:13.723730 3151 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:37:13.733410 3156 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:13.733400 3159 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:13.733400 3157 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:37:13.733621 3151 server_base.cc:1034] running on GCE node
I20260929 10:37:13.734145 3151 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:37:13.734844 3151 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:37:13.736048 3151 hybrid_clock.cc:648] HybridClock initialized: now 1790678233736034 us; error 27 us; skew 500 ppm
I20260929 10:37:13.741454 3151 webserver.cc:460] Webserver started at http://127.15.82.66:39427/ using document root <none> and password file <none>
I20260929 10:37:13.742089 3151 fs_manager.cc:359] Metadata directory not provided
I20260929 10:37:13.742180 3151 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:37:13.742482 3151 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:37:13.744598 3151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790677981592896-15689-0/minicluster-data/ts-1/data/instance:
uuid: "80f9eec44c8f40de884e0c2f12c01ff8"
format_stamp: "Formatted at 2026-09-29 10:37:13 on dist-test-slave-vg89"
server_key: "c0a92fa02e59ba660defd2556a016cf8"
server_key_iv: "43fa49cba053842bf23d93c5d438f24f"
server_key_version: "encryptionkey@0"
I20260929 10:37:13.745169 3151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790677981592896-15689-0/minicluster-data/ts-1/wal/instance:
uuid: "80f9eec44c8f40de884e0c2f12c01ff8"
format_stamp: "Formatted at 2026-09-29 10:37:13 on dist-test-slave-vg89"
server_key: "c0a92fa02e59ba660defd2556a016cf8"
server_key_iv: "43fa49cba053842bf23d93c5d438f24f"
server_key_version: "encryptionkey@0"
I20260929 10:37:13.749112 3151 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260929 10:37:13.751988 3165 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:13.753356 3151 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 10:37:13.753510 3151 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790677981592896-15689-0/minicluster-data/ts-1/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790677981592896-15689-0/minicluster-data/ts-1/wal
uuid: "80f9eec44c8f40de884e0c2f12c01ff8"
format_stamp: "Formatted at 2026-09-29 10:37:13 on dist-test-slave-vg89"
server_key: "c0a92fa02e59ba660defd2556a016cf8"
server_key_iv: "43fa49cba053842bf23d93c5d438f24f"
server_key_version: "encryptionkey@0"
I20260929 10:37:13.753629 3151 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790677981592896-15689-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790677981592896-15689-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790677981592896-15689-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:37:13.767174 3151 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:37:13.767865 3151 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:37:13.768061 3151 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:37:13.768642 3151 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:37:13.769555 3151 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:37:13.769623 3151 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:13.769695 3151 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:37:13.769737 3151 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:13.779137 3151 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.66:45115
I20260929 10:37:13.779170 3278 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.66:45115 every 8 connection(s)
I20260929 10:37:13.780040 3151 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790677981592896-15689-0/minicluster-data/ts-1/data/info.pb
I20260929 10:37:13.787575 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 3151
I20260929 10:37:13.787720 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790677981592896-15689-0/minicluster-data/ts-1/wal/instance
I20260929 10:37:13.788017 15689 external_mini_cluster.cc:1428] Setting key ea83058a0473904c27c5f87f402b46d2
I20260929 10:37:13.789356 2999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:13.781894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:59167 (local address 127.15.82.126:36025)
0929 10:37:13.782030 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:13.782034 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:13.783114 (+ 1080us) server_negotiation.cc:407] Connection header received
0929 10:37:13.784435 (+ 1321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:13.784438 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:13.784490 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:13.784567 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:13.785454 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:13.786006 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:13.787270 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:13.787609 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:13.787727 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:13.788900 (+ 1173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:13.788904 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:13.788906 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:37:13.788918 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:13.788942 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:37:13.788952 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:13.789024 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:13.789153 (+ 129us) server_negotiation.cc:299] Negotiation successful
0929 10:37:13.789206 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260929 10:37:13.790100 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790677981592896-15689-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790677981592896-15689-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790677981592896-15689-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.82.67:0
--local_ip_for_outbound_sockets=127.15.82.67
--webserver_interface=127.15.82.67
--webserver_port=0
--tserver_master_addrs=127.15.82.126:36025
--builtin_ntp_servers=127.15.82.84:43357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260929 10:37:13.790221 3281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:13.782219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:36025 (local address 127.15.82.66:59167)
0929 10:37:13.782958 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:13.783000 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 10:37:13.784221 (+ 1221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:37:13.784730 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:37:13.784736 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:37:13.784847 (+ 111us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:37:13.785321 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:13.785332 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:13.786114 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:13.786116 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:13.787157 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:13.787165 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:13.787780 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:13.787784 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:13.787951 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:13.788693 (+ 742us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 10:37:13.788695 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:37:13.788732 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 10:37:13.788793 (+ 61us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:37:13.789031 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:37:13.789037 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:37:13.789041 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 10:37:13.789129 (+ 88us) client_negotiation.cc:240] Negotiation successful
0929 10:37:13.789398 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":165,"threads_started":1}
I20260929 10:37:13.791424 3279 heartbeater.cc:347] Connected to a master server at 127.15.82.126:36025
I20260929 10:37:13.791653 3279 heartbeater.cc:464] Registering TS with master...
I20260929 10:37:13.792302 3279 heartbeater.cc:511] Master 127.15.82.126:36025 requested a full tablet report, sending...
I20260929 10:37:13.793365 2960 ts_manager.cc:194] Registered new tserver with Master: 80f9eec44c8f40de884e0c2f12c01ff8 (127.15.82.66:45115)
I20260929 10:37:13.793932 2960 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.82.66:59167
W20260929 10:37:13.901955 3282 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:37:13.902293 3282 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:37:13.902369 3282 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:37:13.906562 3282 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:37:13.906630 3282 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:37:13.906718 3282 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.82.67
I20260929 10:37:13.911844 3282 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:43357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790677981592896-15689-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790677981592896-15689-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.82.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.82.67
--webserver_port=0
--tserver_master_addrs=127.15.82.126:36025
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3282
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.82.67
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790677981592896-15689-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:37:13.912947 3282 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:37:13.914294 3282 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 10:37:13.923588 3282 server_base.cc:1034] running on GCE node
W20260929 10:37:13.923568 3287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:13.923555 3290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:13.923547 3288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:37:13.924279 3282 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:37:13.924904 3282 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:37:13.926062 3282 hybrid_clock.cc:648] HybridClock initialized: now 1790678233926026 us; error 50 us; skew 500 ppm
I20260929 10:37:13.927947 3282 webserver.cc:460] Webserver started at http://127.15.82.67:33593/ using document root <none> and password file <none>
I20260929 10:37:13.928592 3282 fs_manager.cc:359] Metadata directory not provided
I20260929 10:37:13.928684 3282 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:37:13.928983 3282 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:37:13.931069 3282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790677981592896-15689-0/minicluster-data/ts-2/data/instance:
uuid: "7b202ba0210d4cae8ab98424eb01cfac"
format_stamp: "Formatted at 2026-09-29 10:37:13 on dist-test-slave-vg89"
server_key: "8a84782348ddb7c1a0ee28f0eb238c78"
server_key_iv: "02f1d9173ff878cf2ab97ff51b7d1692"
server_key_version: "encryptionkey@0"
I20260929 10:37:13.931630 3282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790677981592896-15689-0/minicluster-data/ts-2/wal/instance:
uuid: "7b202ba0210d4cae8ab98424eb01cfac"
format_stamp: "Formatted at 2026-09-29 10:37:13 on dist-test-slave-vg89"
server_key: "8a84782348ddb7c1a0ee28f0eb238c78"
server_key_iv: "02f1d9173ff878cf2ab97ff51b7d1692"
server_key_version: "encryptionkey@0"
I20260929 10:37:13.935307 3282 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260929 10:37:13.937724 3296 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:13.939038 3282 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 10:37:13.939168 3282 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790677981592896-15689-0/minicluster-data/ts-2/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790677981592896-15689-0/minicluster-data/ts-2/wal
uuid: "7b202ba0210d4cae8ab98424eb01cfac"
format_stamp: "Formatted at 2026-09-29 10:37:13 on dist-test-slave-vg89"
server_key: "8a84782348ddb7c1a0ee28f0eb238c78"
server_key_iv: "02f1d9173ff878cf2ab97ff51b7d1692"
server_key_version: "encryptionkey@0"
I20260929 10:37:13.939278 3282 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790677981592896-15689-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790677981592896-15689-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790677981592896-15689-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:37:13.950945 3282 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:37:13.951597 3282 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:37:13.951785 3282 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:37:13.952332 3282 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:37:13.953228 3282 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:37:13.953299 3282 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:13.953370 3282 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:37:13.953426 3282 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:13.962430 3282 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.67:44313
I20260929 10:37:13.962471 3410 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.67:44313 every 8 connection(s)
I20260929 10:37:13.963320 3282 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790677981592896-15689-0/minicluster-data/ts-2/data/info.pb
I20260929 10:37:13.967530 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 3282
I20260929 10:37:13.967659 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790677981592896-15689-0/minicluster-data/ts-2/wal/instance
I20260929 10:37:13.967970 15689 external_mini_cluster.cc:1428] Setting key a0ae520962f79deb8ac402dac109a652
I20260929 10:37:13.973429 2999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:13.965142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.67:34301 (local address 127.15.82.126:36025)
0929 10:37:13.965281 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:13.965285 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:13.966370 (+ 1085us) server_negotiation.cc:407] Connection header received
0929 10:37:13.968147 (+ 1777us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:13.968151 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:13.968199 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:13.968278 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:13.969502 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:13.970265 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:13.971449 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:13.971658 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:13.971747 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:13.972918 (+ 1171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:13.972922 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:13.972924 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:37:13.972936 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:13.972976 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:37:13.972992 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:13.973114 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:13.973263 (+ 149us) server_negotiation.cc:299] Negotiation successful
0929 10:37:13.973314 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260929 10:37:13.974378 3413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:13.965559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:36025 (local address 127.15.82.67:34301)
0929 10:37:13.966220 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:13.966260 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0929 10:37:13.967865 (+ 1605us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:37:13.968496 (+ 631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:37:13.968505 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:37:13.968612 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:37:13.969342 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:13.969366 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:13.970392 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:13.970432 (+ 40us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:13.971329 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:13.971338 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:13.971790 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:13.971793 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:13.971959 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:13.972609 (+ 650us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 10:37:13.972611 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:37:13.972650 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 10:37:13.972711 (+ 61us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:37:13.973109 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:37:13.973117 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:37:13.973130 (+ 13us) client_negotiation.cc:769] Sending connection context
0929 10:37:13.973231 (+ 101us) client_negotiation.cc:240] Negotiation successful
0929 10:37:13.973453 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":113,"threads_started":1}
I20260929 10:37:13.975625 3411 heartbeater.cc:347] Connected to a master server at 127.15.82.126:36025
I20260929 10:37:13.975896 3411 heartbeater.cc:464] Registering TS with master...
I20260929 10:37:13.976418 3411 heartbeater.cc:511] Master 127.15.82.126:36025 requested a full tablet report, sending...
I20260929 10:37:13.977506 2960 ts_manager.cc:194] Registered new tserver with Master: 7b202ba0210d4cae8ab98424eb01cfac (127.15.82.67:44313)
I20260929 10:37:13.978124 2960 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.82.67:34301
I20260929 10:37:13.981668 15689 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 10:37:13.987140 2999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:13.983656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38028 (local address 127.15.82.126:36025)
0929 10:37:13.983806 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:13.983810 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:13.983969 (+ 159us) server_negotiation.cc:407] Connection header received
0929 10:37:13.984046 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:13.984048 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:13.984087 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:13.984146 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:13.984328 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:13.984820 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:13.985633 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:13.985853 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:13.985934 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:13.986712 (+ 778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:13.986715 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:13.986717 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:37:13.986728 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:13.986747 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:37:13.986755 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:13.986828 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:13.986898 (+ 70us) server_negotiation.cc:299] Negotiation successful
0929 10:37:13.986941 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260929 10:37:13.989751 2960 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38028:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 10:37:13.992892 2960 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:37:14.007165 3422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:14.001223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.65:33451 (local address 127.0.0.1:43718)
0929 10:37:14.001725 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:14.001776 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0929 10:37:14.001917 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:37:14.002837 (+ 920us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:37:14.002846 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:37:14.002870 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:37:14.003039 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:14.003052 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:14.004235 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:14.004249 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:14.005544 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:14.005554 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:14.006577 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:14.006580 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:14.006718 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:14.006763 (+ 45us) client_negotiation.cc:769] Sending connection context
0929 10:37:14.006880 (+ 117us) client_negotiation.cc:240] Negotiation successful
0929 10:37:14.007006 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"mutex_wait_us":1,"thread_start_us":77,"threads_started":1}
I20260929 10:37:14.007225 3423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:14.001404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43718 (local address 127.15.82.65:33451)
0929 10:37:14.002232 (+ 828us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:14.002244 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:14.002272 (+ 28us) server_negotiation.cc:407] Connection header received
0929 10:37:14.002358 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:14.002366 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:14.002606 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:14.003224 (+ 618us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:37:14.003384 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:14.004118 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:14.005680 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:14.006449 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:14.006553 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:14.006764 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:14.006936 (+ 172us) server_negotiation.cc:299] Negotiation successful
0929 10:37:14.007051 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":725,"thread_start_us":143,"threads_started":1}
I20260929 10:37:14.007514 3427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:14.002279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43266 (local address 127.15.82.66:45115)
0929 10:37:14.002706 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:14.002720 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:14.002739 (+ 19us) server_negotiation.cc:407] Connection header received
0929 10:37:14.002801 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:14.002805 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:14.002970 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:14.003136 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:37:14.003335 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:14.004506 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:14.005866 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:14.006567 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:14.006978 (+ 411us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:14.007153 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:14.007205 (+ 52us) server_negotiation.cc:299] Negotiation successful
0929 10:37:14.007306 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":136,"threads_started":1}
I20260929 10:37:14.007664 3424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:14.001423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.67:44313 (local address 127.0.0.1:52990)
0929 10:37:14.001823 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:14.001837 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 10:37:14.001933 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:37:14.003291 (+ 1358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:37:14.003296 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:37:14.003317 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:37:14.003429 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:14.003436 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:14.004866 (+ 1430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:14.004870 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:14.005961 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:14.005971 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:14.007043 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:14.007046 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:14.007139 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:14.007161 (+ 22us) client_negotiation.cc:769] Sending connection context
0929 10:37:14.007209 (+ 48us) client_negotiation.cc:240] Negotiation successful
0929 10:37:14.007286 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"mutex_wait_us":3,"thread_start_us":136,"threads_started":1}
I20260929 10:37:14.007992 3425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:14.001918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.66:45115 (local address 127.0.0.1:43266)
0929 10:37:14.002217 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:14.002231 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 10:37:14.002322 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:37:14.003116 (+ 794us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:37:14.003122 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:37:14.003143 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:37:14.003236 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:14.003251 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:14.004627 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:14.004631 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:14.005738 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:14.005759 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:14.006699 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:14.006704 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:14.006773 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:14.006795 (+ 22us) client_negotiation.cc:769] Sending connection context
0929 10:37:14.006855 (+ 60us) client_negotiation.cc:240] Negotiation successful
0929 10:37:14.007896 (+ 1041us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"mutex_wait_us":939,"thread_start_us":86,"threads_started":1}
I20260929 10:37:14.008385 3426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:14.001999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52990 (local address 127.15.82.67:44313)
0929 10:37:14.002608 (+ 609us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:14.002620 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:14.002645 (+ 25us) server_negotiation.cc:407] Connection header received
0929 10:37:14.002722 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:14.002727 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:14.003137 (+ 410us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:14.003577 (+ 440us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:37:14.003712 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:14.004750 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:14.006088 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:14.006908 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:14.007930 (+ 1022us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:14.008091 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:14.008135 (+ 44us) server_negotiation.cc:299] Negotiation successful
0929 10:37:14.008223 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":499,"thread_start_us":99,"threads_started":1}
I20260929 10:37:14.010656 3213 tablet_service.cc:1467] Processing CreateTablet for tablet 7c1b0e2503c54bf998eec632ec871f45 (DEFAULT_TABLE table=test-table [id=d54ce8b885f84bd285af1d0ae13a0308]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:37:14.010629 3082 tablet_service.cc:1467] Processing CreateTablet for tablet 7c1b0e2503c54bf998eec632ec871f45 (DEFAULT_TABLE table=test-table [id=d54ce8b885f84bd285af1d0ae13a0308]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:37:14.010799 3344 tablet_service.cc:1467] Processing CreateTablet for tablet 7c1b0e2503c54bf998eec632ec871f45 (DEFAULT_TABLE table=test-table [id=d54ce8b885f84bd285af1d0ae13a0308]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:37:14.011633 3213 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c1b0e2503c54bf998eec632ec871f45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:37:14.011651 3344 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c1b0e2503c54bf998eec632ec871f45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:37:14.011651 3082 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c1b0e2503c54bf998eec632ec871f45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:37:14.018572 3428 tablet_bootstrap.cc:492] T 7c1b0e2503c54bf998eec632ec871f45 P 80f9eec44c8f40de884e0c2f12c01ff8: Bootstrap starting.
I20260929 10:37:14.019402 3429 tablet_bootstrap.cc:492] T 7c1b0e2503c54bf998eec632ec871f45 P 08bfd5769d6f49f89be30c103e81b9af: Bootstrap starting.
I20260929 10:37:14.019476 3430 tablet_bootstrap.cc:492] T 7c1b0e2503c54bf998eec632ec871f45 P 7b202ba0210d4cae8ab98424eb01cfac: Bootstrap starting.
I20260929 10:37:14.020694 3428 tablet_bootstrap.cc:654] T 7c1b0e2503c54bf998eec632ec871f45 P 80f9eec44c8f40de884e0c2f12c01ff8: Neither blocks nor log segments found. Creating new log.
I20260929 10:37:14.021435 3428 log.cc:824] T 7c1b0e2503c54bf998eec632ec871f45 P 80f9eec44c8f40de884e0c2f12c01ff8: Log is configured to *not* fsync() on all Append() calls
I20260929 10:37:14.021603 3430 tablet_bootstrap.cc:654] T 7c1b0e2503c54bf998eec632ec871f45 P 7b202ba0210d4cae8ab98424eb01cfac: Neither blocks nor log segments found. Creating new log.
I20260929 10:37:14.021603 3429 tablet_bootstrap.cc:654] T 7c1b0e2503c54bf998eec632ec871f45 P 08bfd5769d6f49f89be30c103e81b9af: Neither blocks nor log segments found. Creating new log.
I20260929 10:37:14.022459 3430 log.cc:824] T 7c1b0e2503c54bf998eec632ec871f45 P 7b202ba0210d4cae8ab98424eb01cfac: Log is configured to *not* fsync() on all Append() calls
I20260929 10:37:14.022459 3429 log.cc:824] T 7c1b0e2503c54bf998eec632ec871f45 P 08bfd5769d6f49f89be30c103e81b9af: Log is configured to *not* fsync() on all Append() calls
I20260929 10:37:14.023576 3428 tablet_bootstrap.cc:492] T 7c1b0e2503c54bf998eec632ec871f45 P 80f9eec44c8f40de884e0c2f12c01ff8: No bootstrap required, opened a new log
I20260929 10:37:14.023773 3428 ts_tablet_manager.cc:1397] T 7c1b0e2503c54bf998eec632ec871f45 P 80f9eec44c8f40de884e0c2f12c01ff8: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260929 10:37:14.024289 3429 tablet_bootstrap.cc:492] T 7c1b0e2503c54bf998eec632ec871f45 P 08bfd5769d6f49f89be30c103e81b9af: No bootstrap required, opened a new log
I20260929 10:37:14.024484 3429 ts_tablet_manager.cc:1397] T 7c1b0e2503c54bf998eec632ec871f45 P 08bfd5769d6f49f89be30c103e81b9af: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260929 10:37:14.024492 3430 tablet_bootstrap.cc:492] T 7c1b0e2503c54bf998eec632ec871f45 P 7b202ba0210d4cae8ab98424eb01cfac: No bootstrap required, opened a new log
I20260929 10:37:14.024662 3430 ts_tablet_manager.cc:1397] T 7c1b0e2503c54bf998eec632ec871f45 P 7b202ba0210d4cae8ab98424eb01cfac: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260929 10:37:14.027729 3429 raft_consensus.cc:348] T 7c1b0e2503c54bf998eec632ec871f45 P 08bfd5769d6f49f89be30c103e81b9af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08bfd5769d6f49f89be30c103e81b9af" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 33451 } } peers { permanent_uuid: "7b202ba0210d4cae8ab98424eb01cfac" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 44313 } } peers { permanent_uuid: "80f9eec44c8f40de884e0c2f12c01ff8" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 45115 } }
I20260929 10:37:14.027967 3429 raft_consensus.cc:374] T 7c1b0e2503c54bf998eec632ec871f45 P 08bfd5769d6f49f89be30c103e81b9af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:37:14.028018 3429 raft_consensus.cc:729] T 7c1b0e2503c54bf998eec632ec871f45 P 08bfd5769d6f49f89be30c103e81b9af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08bfd5769d6f49f89be30c103e81b9af, State: Initialized, Role: FOLLOWER
I20260929 10:37:14.028236 3428 raft_consensus.cc:348] T 7c1b0e2503c54bf998eec632ec871f45 P 80f9eec44c8f40de884e0c2f12c01ff8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08bfd5769d6f49f89be30c103e81b9af" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 33451 } } peers { permanent_uuid: "7b202ba0210d4cae8ab98424eb01cfac" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 44313 } } peers { permanent_uuid: "80f9eec44c8f40de884e0c2f12c01ff8" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 45115 } }
I20260929 10:37:14.028496 3428 raft_consensus.cc:374] T 7c1b0e2503c54bf998eec632ec871f45 P 80f9eec44c8f40de884e0c2f12c01ff8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:37:14.028553 3428 raft_consensus.cc:729] T 7c1b0e2503c54bf998eec632ec871f45 P 80f9eec44c8f40de884e0c2f12c01ff8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80f9eec44c8f40de884e0c2f12c01ff8, State: Initialized, Role: FOLLOWER
I20260929 10:37:14.028494 3429 consensus_queue.cc:260] T 7c1b0e2503c54bf998eec632ec871f45 P 08bfd5769d6f49f89be30c103e81b9af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08bfd5769d6f49f89be30c103e81b9af" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 33451 } } peers { permanent_uuid: "7b202ba0210d4cae8ab98424eb01cfac" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 44313 } } peers { permanent_uuid: "80f9eec44c8f40de884e0c2f12c01ff8" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 45115 } }
I20260929 10:37:14.028797 3430 raft_consensus.cc:348] T 7c1b0e2503c54bf998eec632ec871f45 P 7b202ba0210d4cae8ab98424eb01cfac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08bfd5769d6f49f89be30c103e81b9af" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 33451 } } peers { permanent_uuid: "7b202ba0210d4cae8ab98424eb01cfac" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 44313 } } peers { permanent_uuid: "80f9eec44c8f40de884e0c2f12c01ff8" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 45115 } }
I20260929 10:37:14.028945 3428 consensus_queue.cc:260] T 7c1b0e2503c54bf998eec632ec871f45 P 80f9eec44c8f40de884e0c2f12c01ff8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08bfd5769d6f49f89be30c103e81b9af" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 33451 } } peers { permanent_uuid: "7b202ba0210d4cae8ab98424eb01cfac" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 44313 } } peers { permanent_uuid: "80f9eec44c8f40de884e0c2f12c01ff8" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 45115 } }
I20260929 10:37:14.029024 3430 raft_consensus.cc:374] T 7c1b0e2503c54bf998eec632ec871f45 P 7b202ba0210d4cae8ab98424eb01cfac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:37:14.029078 3430 raft_consensus.cc:729] T 7c1b0e2503c54bf998eec632ec871f45 P 7b202ba0210d4cae8ab98424eb01cfac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b202ba0210d4cae8ab98424eb01cfac, State: Initialized, Role: FOLLOWER
I20260929 10:37:14.029438 3430 consensus_queue.cc:260] T 7c1b0e2503c54bf998eec632ec871f45 P 7b202ba0210d4cae8ab98424eb01cfac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08bfd5769d6f49f89be30c103e81b9af" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 33451 } } peers { permanent_uuid: "7b202ba0210d4cae8ab98424eb01cfac" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 44313 } } peers { permanent_uuid: "80f9eec44c8f40de884e0c2f12c01ff8" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 45115 } }
I20260929 10:37:14.029560 3429 ts_tablet_manager.cc:1428] T 7c1b0e2503c54bf998eec632ec871f45 P 08bfd5769d6f49f89be30c103e81b9af: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260929 10:37:14.029844 3148 heartbeater.cc:503] Master 127.15.82.126:36025 was elected leader, sending a full tablet report...
I20260929 10:37:14.029845 3279 heartbeater.cc:503] Master 127.15.82.126:36025 was elected leader, sending a full tablet report...
I20260929 10:37:14.029896 3428 ts_tablet_manager.cc:1428] T 7c1b0e2503c54bf998eec632ec871f45 P 80f9eec44c8f40de884e0c2f12c01ff8: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260929 10:37:14.030336 3430 ts_tablet_manager.cc:1428] T 7c1b0e2503c54bf998eec632ec871f45 P 7b202ba0210d4cae8ab98424eb01cfac: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260929 10:37:14.030707 3411 heartbeater.cc:503] Master 127.15.82.126:36025 was elected leader, sending a full tablet report...
W20260929 10:37:14.030999 3280 tablet.cc:2369] T 7c1b0e2503c54bf998eec632ec871f45 P 80f9eec44c8f40de884e0c2f12c01ff8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 10:37:14.060523 3435 raft_consensus.cc:482] T 7c1b0e2503c54bf998eec632ec871f45 P 80f9eec44c8f40de884e0c2f12c01ff8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:37:14.060778 3435 raft_consensus.cc:504] T 7c1b0e2503c54bf998eec632ec871f45 P 80f9eec44c8f40de884e0c2f12c01ff8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08bfd5769d6f49f89be30c103e81b9af" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 33451 } } peers { permanent_uuid: "7b202ba0210d4cae8ab98424eb01cfac" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 44313 } } peers { permanent_uuid: "80f9eec44c8f40de884e0c2f12c01ff8" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 45115 } }
I20260929 10:37:14.061985 3435 leader_election.cc:288] T 7c1b0e2503c54bf998eec632ec871f45 P 80f9eec44c8f40de884e0c2f12c01ff8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 08bfd5769d6f49f89be30c103e81b9af (127.15.82.65:33451), 7b202ba0210d4cae8ab98424eb01cfac (127.15.82.67:44313)
I20260929 10:37:14.066195 3426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:14.062583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:54195 (local address 127.15.82.67:44313)
0929 10:37:14.062716 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:14.062721 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:14.062737 (+ 16us) server_negotiation.cc:407] Connection header received
0929 10:37:14.062814 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:14.062820 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:14.062883 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:14.062966 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:37:14.063306 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:14.063809 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:14.065080 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:14.065668 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:14.065758 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:14.065796 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:14.066033 (+ 237us) server_negotiation.cc:299] Negotiation successful
0929 10:37:14.066093 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260929 10:37:14.066219 3281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:14.062275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.67:44313 (local address 127.15.82.66:54195)
0929 10:37:14.062524 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:14.062540 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 10:37:14.062631 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:37:14.063004 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:37:14.063008 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:37:14.063030 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:37:14.063162 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:14.063170 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:14.063922 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:14.063925 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:14.064948 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:14.064956 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:14.065792 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:14.065795 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:14.065898 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:14.065930 (+ 32us) client_negotiation.cc:769] Sending connection context
0929 10:37:14.066011 (+ 81us) client_negotiation.cc:240] Negotiation successful
0929 10:37:14.066091 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156}
I20260929 10:37:14.066965 3437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:14.062275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.65:33451 (local address 127.15.82.66:42669)
0929 10:37:14.062948 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:14.062963 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 10:37:14.063096 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:37:14.063465 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:37:14.063468 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:37:14.063484 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:37:14.063612 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:14.063616 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:14.064582 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:14.064584 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:14.065533 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:14.065543 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:14.066612 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:14.066616 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:14.066729 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:14.066751 (+ 22us) client_negotiation.cc:769] Sending connection context
0929 10:37:14.066815 (+ 64us) client_negotiation.cc:240] Negotiation successful
0929 10:37:14.066859 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":586,"mutex_wait_us":318,"thread_start_us":131,"threads_started":1}
I20260929 10:37:14.067054 3423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:14.062418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.82.66:42669 (local address 127.15.82.65:33451)
0929 10:37:14.062571 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:14.062576 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:14.063024 (+ 448us) server_negotiation.cc:407] Connection header received
0929 10:37:14.063293 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:14.063298 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:14.063362 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:14.063453 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:37:14.063728 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:14.064472 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:14.065682 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:14.066322 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:14.066486 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:14.066535 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:14.066860 (+ 325us) server_negotiation.cc:299] Negotiation successful
0929 10:37:14.066935 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260929 10:37:14.066998 3364 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c1b0e2503c54bf998eec632ec871f45" candidate_uuid: "80f9eec44c8f40de884e0c2f12c01ff8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b202ba0210d4cae8ab98424eb01cfac" is_pre_election: true
I20260929 10:37:14.067348 3364 raft_consensus.cc:2454] T 7c1b0e2503c54bf998eec632ec871f45 P 7b202ba0210d4cae8ab98424eb01cfac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80f9eec44c8f40de884e0c2f12c01ff8 in term 0.
I20260929 10:37:14.067843 3102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c1b0e2503c54bf998eec632ec871f45" candidate_uuid: "80f9eec44c8f40de884e0c2f12c01ff8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08bfd5769d6f49f89be30c103e81b9af" is_pre_election: true
I20260929 10:37:14.067984 3169 leader_election.cc:302] T 7c1b0e2503c54bf998eec632ec871f45 P 80f9eec44c8f40de884e0c2f12c01ff8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7b202ba0210d4cae8ab98424eb01cfac, 80f9eec44c8f40de884e0c2f12c01ff8; no voters:
I20260929 10:37:14.068150 3102 raft_consensus.cc:2454] T 7c1b0e2503c54bf998eec632ec871f45 P 08bfd5769d6f49f89be30c103e81b9af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80f9eec44c8f40de884e0c2f12c01ff8 in term 0.
I20260929 10:37:14.068292 3435 raft_consensus.cc:2781] T 7c1b0e2503c54bf998eec632ec871f45 P 80f9eec44c8f40de884e0c2f12c01ff8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:37:14.068397 3435 raft_consensus.cc:482] T 7c1b0e2503c54bf998eec632ec871f45 P 80f9eec44c8f40de884e0c2f12c01ff8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:37:14.068456 3435 raft_consensus.cc:3037] T 7c1b0e2503c54bf998eec632ec871f45 P 80f9eec44c8f40de884e0c2f12c01ff8 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:37:14.069518 3435 raft_consensus.cc:504] T 7c1b0e2503c54bf998eec632ec871f45 P 80f9eec44c8f40de884e0c2f12c01ff8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08bfd5769d6f49f89be30c103e81b9af" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 33451 } } peers { permanent_uuid: "7b202ba0210d4cae8ab98424eb01cfac" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 44313 } } peers { permanent_uuid: "80f9eec44c8f40de884e0c2f12c01ff8" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 45115 } }
I20260929 10:37:14.069880 3435 leader_election.cc:288] T 7c1b0e2503c54bf998eec632ec871f45 P 80f9eec44c8f40de884e0c2f12c01ff8 [CANDIDATE]: Term 1 election: Requested vote from peers 08bfd5769d6f49f89be30c103e81b9af (127.15.82.65:33451), 7b202ba0210d4cae8ab98424eb01cfac (127.15.82.67:44313)
I20260929 10:37:14.070176 3102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c1b0e2503c54bf998eec632ec871f45" candidate_uuid: "80f9eec44c8f40de884e0c2f12c01ff8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08bfd5769d6f49f89be30c103e81b9af"
I20260929 10:37:14.070235 3364 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c1b0e2503c54bf998eec632ec871f45" candidate_uuid: "80f9eec44c8f40de884e0c2f12c01ff8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b202ba0210d4cae8ab98424eb01cfac"
I20260929 10:37:14.070312 3102 raft_consensus.cc:3037] T 7c1b0e2503c54bf998eec632ec871f45 P 08bfd5769d6f49f89be30c103e81b9af [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:37:14.070359 3364 raft_consensus.cc:3037] T 7c1b0e2503c54bf998eec632ec871f45 P 7b202ba0210d4cae8ab98424eb01cfac [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:37:14.071488 3102 raft_consensus.cc:2454] T 7c1b0e2503c54bf998eec632ec871f45 P 08bfd5769d6f49f89be30c103e81b9af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80f9eec44c8f40de884e0c2f12c01ff8 in term 1.
I20260929 10:37:14.071611 3364 raft_consensus.cc:2454] T 7c1b0e2503c54bf998eec632ec871f45 P 7b202ba0210d4cae8ab98424eb01cfac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80f9eec44c8f40de884e0c2f12c01ff8 in term 1.
I20260929 10:37:14.071801 3167 leader_election.cc:302] T 7c1b0e2503c54bf998eec632ec871f45 P 80f9eec44c8f40de884e0c2f12c01ff8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08bfd5769d6f49f89be30c103e81b9af, 80f9eec44c8f40de884e0c2f12c01ff8; no voters:
I20260929 10:37:14.071995 3435 raft_consensus.cc:2781] T 7c1b0e2503c54bf998eec632ec871f45 P 80f9eec44c8f40de884e0c2f12c01ff8 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:37:14.072290 3435 raft_consensus.cc:686] T 7c1b0e2503c54bf998eec632ec871f45 P 80f9eec44c8f40de884e0c2f12c01ff8 [term 1 LEADER]: Becoming Leader. State: Replica: 80f9eec44c8f40de884e0c2f12c01ff8, State: Running, Role: LEADER
I20260929 10:37:14.072682 3435 consensus_queue.cc:237] T 7c1b0e2503c54bf998eec632ec871f45 P 80f9eec44c8f40de884e0c2f12c01ff8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08bfd5769d6f49f89be30c103e81b9af" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 33451 } } peers { permanent_uuid: "7b202ba0210d4cae8ab98424eb01cfac" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 44313 } } peers { permanent_uuid: "80f9eec44c8f40de884e0c2f12c01ff8" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 45115 } }
I20260929 10:37:14.076004 2958 catalog_manager.cc:5495] T 7c1b0e2503c54bf998eec632ec871f45 P 80f9eec44c8f40de884e0c2f12c01ff8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 80f9eec44c8f40de884e0c2f12c01ff8 (127.15.82.66). New cstate: current_term: 1 leader_uuid: "80f9eec44c8f40de884e0c2f12c01ff8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08bfd5769d6f49f89be30c103e81b9af" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 33451 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7b202ba0210d4cae8ab98424eb01cfac" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 44313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "80f9eec44c8f40de884e0c2f12c01ff8" member_type: VOTER last_known_addr { host: "127.15.82.66" port: 45115 } health_report { overall_health: HEALTHY } } }
I20260929 10:37:14.086546 3427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:14.083108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43280 (local address 127.15.82.66:45115)
0929 10:37:14.083217 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:14.083228 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:14.083240 (+ 12us) server_negotiation.cc:407] Connection header received
0929 10:37:14.083400 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:14.083402 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:14.083454 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:14.083536 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:37:14.083753 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:14.084238 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:14.085086 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:14.085319 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:14.085400 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:14.085712 (+ 312us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:37:14.086116 (+ 404us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:37:14.086203 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:14.086312 (+ 109us) server_negotiation.cc:299] Negotiation successful
0929 10:37:14.086355 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260929 10:37:14.092293 3364 raft_consensus.cc:1261] T 7c1b0e2503c54bf998eec632ec871f45 P 7b202ba0210d4cae8ab98424eb01cfac [term 1 FOLLOWER]: Refusing update from remote peer 80f9eec44c8f40de884e0c2f12c01ff8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:37:14.092291 3102 raft_consensus.cc:1261] T 7c1b0e2503c54bf998eec632ec871f45 P 08bfd5769d6f49f89be30c103e81b9af [term 1 FOLLOWER]: Refusing update from remote peer 80f9eec44c8f40de884e0c2f12c01ff8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:37:14.092952 3435 consensus_queue.cc:1035] T 7c1b0e2503c54bf998eec632ec871f45 P 80f9eec44c8f40de884e0c2f12c01ff8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b202ba0210d4cae8ab98424eb01cfac" member_type: VOTER last_known_addr { host: "127.15.82.67" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:37:14.093196 3438 consensus_queue.cc:1035] T 7c1b0e2503c54bf998eec632ec871f45 P 80f9eec44c8f40de884e0c2f12c01ff8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "08bfd5769d6f49f89be30c103e81b9af" member_type: VOTER last_known_addr { host: "127.15.82.65" port: 33451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260929 10:37:14.094099 3149 tablet.cc:2369] T 7c1b0e2503c54bf998eec632ec871f45 P 08bfd5769d6f49f89be30c103e81b9af: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 10:37:14.100203 3443 mvcc.cc:205] Tried to move back new op lower bound from 7334618046835060736 to 7334618046764879872. Current Snapshot: MvccSnapshot[applied={T|T < 7334618046835060736}]
I20260929 10:37:14.101850 3442 mvcc.cc:205] Tried to move back new op lower bound from 7334618046835060736 to 7334618046764879872. Current Snapshot: MvccSnapshot[applied={T|T < 7334618046835060736}]
I20260929 10:37:14.109853 2958 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38028:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:37:14.110039 2958 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38028:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:37:14.112520 2958 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 57c9a8cb43584e139991cb78a786b6e2: Sending DeleteTablet for 3 replicas of tablet 7c1b0e2503c54bf998eec632ec871f45
I20260929 10:37:14.113278 3082 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c1b0e2503c54bf998eec632ec871f45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:37:14 UTC) from {username='slave'} at 127.0.0.1:43718
I20260929 10:37:14.113284 3344 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c1b0e2503c54bf998eec632ec871f45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:37:14 UTC) from {username='slave'} at 127.0.0.1:52990
I20260929 10:37:14.113379 3213 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c1b0e2503c54bf998eec632ec871f45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:37:14 UTC) from {username='slave'} at 127.0.0.1:43266
I20260929 10:37:14.113751 3451 tablet_replica.cc:330] T 7c1b0e2503c54bf998eec632ec871f45 P 7b202ba0210d4cae8ab98424eb01cfac: stopping tablet replica
I20260929 10:37:14.114091 3450 tablet_replica.cc:330] T 7c1b0e2503c54bf998eec632ec871f45 P 08bfd5769d6f49f89be30c103e81b9af: stopping tablet replica
I20260929 10:37:14.114338 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 3020
I20260929 10:37:14.116079 3452 tablet_replica.cc:330] T 7c1b0e2503c54bf998eec632ec871f45 P 80f9eec44c8f40de884e0c2f12c01ff8: stopping tablet replica
I20260929 10:37:14.116410 3452 raft_consensus.cc:2229] T 7c1b0e2503c54bf998eec632ec871f45 P 80f9eec44c8f40de884e0c2f12c01ff8 [term 1 LEADER]: Raft consensus shutting down.
I20260929 10:37:14.116865 3452 raft_consensus.cc:2258] T 7c1b0e2503c54bf998eec632ec871f45 P 80f9eec44c8f40de884e0c2f12c01ff8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:37:14.118520 3452 ts_tablet_manager.cc:1905] T 7c1b0e2503c54bf998eec632ec871f45 P 80f9eec44c8f40de884e0c2f12c01ff8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:37:14.122118 3452 ts_tablet_manager.cc:1918] T 7c1b0e2503c54bf998eec632ec871f45 P 80f9eec44c8f40de884e0c2f12c01ff8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:37:14.122225 3452 log.cc:1196] T 7c1b0e2503c54bf998eec632ec871f45 P 80f9eec44c8f40de884e0c2f12c01ff8: Deleting WAL directory at /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790677981592896-15689-0/minicluster-data/ts-1/wal/wals/7c1b0e2503c54bf998eec632ec871f45
I20260929 10:37:14.122511 3451 raft_consensus.cc:2229] T 7c1b0e2503c54bf998eec632ec871f45 P 7b202ba0210d4cae8ab98424eb01cfac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:37:14.122598 3452 ts_tablet_manager.cc:1939] T 7c1b0e2503c54bf998eec632ec871f45 P 80f9eec44c8f40de884e0c2f12c01ff8: Deleting consensus metadata
I20260929 10:37:14.122972 3451 raft_consensus.cc:2258] T 7c1b0e2503c54bf998eec632ec871f45 P 7b202ba0210d4cae8ab98424eb01cfac [term 1 FOLLOWER]: Raft consensus is shut down!
W20260929 10:37:14.123271 2945 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.82.65:33451 (error 108)
I20260929 10:37:14.124132 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 3151
I20260929 10:37:14.124745 2945 catalog_manager.cc:4841] TS 80f9eec44c8f40de884e0c2f12c01ff8 (127.15.82.66:45115): tablet 7c1b0e2503c54bf998eec632ec871f45 (table test-table [id=d54ce8b885f84bd285af1d0ae13a0308]) successfully deleted
I20260929 10:37:14.126214 3451 ts_tablet_manager.cc:1905] T 7c1b0e2503c54bf998eec632ec871f45 P 7b202ba0210d4cae8ab98424eb01cfac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:37:14.126315 3425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:14.125935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.65:33451 (local address 127.0.0.1:43720)
0929 10:37:14.126107 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:14.126196 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.65:33451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260929 10:37:14.126529 2945 catalog_manager.cc:4584] TS 08bfd5769d6f49f89be30c103e81b9af (127.15.82.65:33451): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7c1b0e2503c54bf998eec632ec871f45: Network error: Client connection negotiation failed: client connection to 127.15.82.65:33451: connect: Connection refused (error 111)
I20260929 10:37:14.128638 3451 ts_tablet_manager.cc:1918] T 7c1b0e2503c54bf998eec632ec871f45 P 7b202ba0210d4cae8ab98424eb01cfac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:37:14.128759 3451 log.cc:1196] T 7c1b0e2503c54bf998eec632ec871f45 P 7b202ba0210d4cae8ab98424eb01cfac: Deleting WAL directory at /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790677981592896-15689-0/minicluster-data/ts-2/wal/wals/7c1b0e2503c54bf998eec632ec871f45
I20260929 10:37:14.129041 3451 ts_tablet_manager.cc:1939] T 7c1b0e2503c54bf998eec632ec871f45 P 7b202ba0210d4cae8ab98424eb01cfac: Deleting consensus metadata
I20260929 10:37:14.130393 2947 catalog_manager.cc:4841] TS 7b202ba0210d4cae8ab98424eb01cfac (127.15.82.67:44313): tablet 7c1b0e2503c54bf998eec632ec871f45 (table test-table [id=d54ce8b885f84bd285af1d0ae13a0308]) successfully deleted
I20260929 10:37:14.130986 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 3282
I20260929 10:37:14.136999 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 2929
2026-09-29T10:37:14Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (990 ms)
[----------] 6 tests from EncryptionPolicyTest (10498 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790677981592896-15689-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:37:14 dist-test-slave-vg89 krb5kdc[3456](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:37:14 dist-test-slave-vg89 krb5kdc[3456](info): set up 2 sockets
Sep 29 10:37:14 dist-test-slave-vg89 krb5kdc[3456](info): commencing operation
krb5kdc: starting...
W20260929 10:37:15.190990 15689 mini_kdc.cc:121] Time spent starting KDC: real 1.032s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Sep 29 10:37:15 dist-test-slave-vg89 krb5kdc[3456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678235, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:37:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:37:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790677981592896-15689-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790677981592896-15689-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 10:37:15.314939 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790677981592896-15689-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790677981592896-15689-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790677981592896-15689-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43329
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790677981592896-15689-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43329
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:37:15.428996 3469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:37:15.429332 3469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:37:15.429415 3469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:37:15.433511 3469 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:37:15.433602 3469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:37:15.433640 3469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:37:15.433694 3469 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:37:15.433728 3469 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:37:15.438992 3469 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790677981592896-15689-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790677981592896-15689-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43329
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790677981592896-15689-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43329
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3469
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790677981592896-15689-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:37:15.440057 3469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:37:15.441131 3469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:37:15.449541 3475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:15.449537 3477 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:37:15.449709 3469 server_base.cc:1034] running on GCE node
W20260929 10:37:15.449517 3474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:37:15.450498 3469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:37:15.451562 3469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:37:15.452728 3469 hybrid_clock.cc:648] HybridClock initialized: now 1790678235452703 us; error 37 us; skew 500 ppm
Sep 29 10:37:15 dist-test-slave-vg89 krb5kdc[3456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678235, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:37:15.455595 3469 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 10:37:15.456466 3469 webserver.cc:460] Webserver started at http://127.0.0.1:34061/ using document root <none> and password file <none>
I20260929 10:37:15.456976 3469 fs_manager.cc:359] Metadata directory not provided
I20260929 10:37:15.457046 3469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:37:15.457223 3469 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:37:15.459276 3469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790677981592896-15689-0/minicluster-data/master-0/data/instance:
uuid: "ca70b18c31fe4e278ac3420cde483c70"
format_stamp: "Formatted at 2026-09-29 10:37:15 on dist-test-slave-vg89"
server_key: "d2b6c78a43e89a75dd96cf533924dcdf"
server_key_iv: "da69fa85cf4910fd34149e278f3b3216"
server_key_version: "encryptionkey@0"
I20260929 10:37:15.459797 3469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790677981592896-15689-0/minicluster-data/master-0/wal/instance:
uuid: "ca70b18c31fe4e278ac3420cde483c70"
format_stamp: "Formatted at 2026-09-29 10:37:15 on dist-test-slave-vg89"
server_key: "d2b6c78a43e89a75dd96cf533924dcdf"
server_key_iv: "da69fa85cf4910fd34149e278f3b3216"
server_key_version: "encryptionkey@0"
I20260929 10:37:15.463375 3469 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260929 10:37:15.465718 3484 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:15.467110 3469 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 10:37:15.467249 3469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790677981592896-15689-0/minicluster-data/master-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790677981592896-15689-0/minicluster-data/master-0/wal
uuid: "ca70b18c31fe4e278ac3420cde483c70"
format_stamp: "Formatted at 2026-09-29 10:37:15 on dist-test-slave-vg89"
server_key: "d2b6c78a43e89a75dd96cf533924dcdf"
server_key_iv: "da69fa85cf4910fd34149e278f3b3216"
server_key_version: "encryptionkey@0"
I20260929 10:37:15.467367 3469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790677981592896-15689-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790677981592896-15689-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790677981592896-15689-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:37:15.478914 3469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:37:15.481932 3469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:37:15.482133 3469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:37:15.489459 3469 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43329
I20260929 10:37:15.489501 3536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43329 every 8 connection(s)
I20260929 10:37:15.490536 3469 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
I20260929 10:37:15.491546 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 3469
I20260929 10:37:15.491698 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790677981592896-15689-0/minicluster-data/master-0/wal/instance
I20260929 10:37:15.492017 15689 external_mini_cluster.cc:1428] Setting key f89ceda069c2b05ff7bce579130ef6f5
I20260929 10:37:15.493893 3537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:37:15.501137 3537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca70b18c31fe4e278ac3420cde483c70: Bootstrap starting.
Sep 29 10:37:15 dist-test-slave-vg89 krb5kdc[3456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678235, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260929 10:37:15.504242 3537 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ca70b18c31fe4e278ac3420cde483c70: Neither blocks nor log segments found. Creating new log.
I20260929 10:37:15.505228 3537 log.cc:824] T 00000000000000000000000000000000 P ca70b18c31fe4e278ac3420cde483c70: Log is configured to *not* fsync() on all Append() calls
I20260929 10:37:15.507647 3537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca70b18c31fe4e278ac3420cde483c70: No bootstrap required, opened a new log
I20260929 10:37:15.508549 3540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:15.493467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41847 (local address 127.0.0.1:43329)
0929 10:37:15.493888 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:15.493903 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:15.493959 (+ 56us) server_negotiation.cc:407] Connection header received
0929 10:37:15.494681 (+ 722us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:15.494699 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:15.495037 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:15.495310 (+ 273us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:15.496293 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:15.497455 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:15.498747 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:15.499223 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:15.499335 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:15.502326 (+ 2991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:15.502335 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:15.502355 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:37:15.502431 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:15.504607 (+ 2176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:15.505120 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:15.505126 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:15.505132 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:15.505299 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:15.505718 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:15.505722 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:15.505724 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:15.506249 (+ 525us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:15.506562 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:15.507008 (+ 446us) server_negotiation.cc:299] Negotiation successful
0929 10:37:15.507248 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":153,"threads_started":1}
I20260929 10:37:15.511572 15689 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 10:37:15.511721 3537 raft_consensus.cc:348] T 00000000000000000000000000000000 P ca70b18c31fe4e278ac3420cde483c70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca70b18c31fe4e278ac3420cde483c70" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43329 } }
I20260929 10:37:15.511974 3537 raft_consensus.cc:374] T 00000000000000000000000000000000 P ca70b18c31fe4e278ac3420cde483c70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:37:15.512075 3537 raft_consensus.cc:729] T 00000000000000000000000000000000 P ca70b18c31fe4e278ac3420cde483c70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca70b18c31fe4e278ac3420cde483c70, State: Initialized, Role: FOLLOWER
I20260929 10:37:15.512521 3537 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca70b18c31fe4e278ac3420cde483c70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca70b18c31fe4e278ac3420cde483c70" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43329 } }
I20260929 10:37:15.512688 3537 raft_consensus.cc:388] T 00000000000000000000000000000000 P ca70b18c31fe4e278ac3420cde483c70 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:37:15.512794 3537 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca70b18c31fe4e278ac3420cde483c70 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:37:15.512912 3537 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ca70b18c31fe4e278ac3420cde483c70 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:37:15.513972 3537 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca70b18c31fe4e278ac3420cde483c70 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca70b18c31fe4e278ac3420cde483c70" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43329 } }
I20260929 10:37:15.514384 3537 leader_election.cc:302] T 00000000000000000000000000000000 P ca70b18c31fe4e278ac3420cde483c70 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ca70b18c31fe4e278ac3420cde483c70; no voters:
I20260929 10:37:15.514856 3537 leader_election.cc:288] T 00000000000000000000000000000000 P ca70b18c31fe4e278ac3420cde483c70 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:37:15.514910 3547 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ca70b18c31fe4e278ac3420cde483c70 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:37:15.515154 3547 raft_consensus.cc:686] T 00000000000000000000000000000000 P ca70b18c31fe4e278ac3420cde483c70 [term 1 LEADER]: Becoming Leader. State: Replica: ca70b18c31fe4e278ac3420cde483c70, State: Running, Role: LEADER
I20260929 10:37:15.515447 3547 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca70b18c31fe4e278ac3420cde483c70 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca70b18c31fe4e278ac3420cde483c70" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43329 } }
I20260929 10:37:15.516232 3537 sys_catalog.cc:564] T 00000000000000000000000000000000 P ca70b18c31fe4e278ac3420cde483c70 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:37:15.516533 3549 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca70b18c31fe4e278ac3420cde483c70 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca70b18c31fe4e278ac3420cde483c70. Latest consensus state: current_term: 1 leader_uuid: "ca70b18c31fe4e278ac3420cde483c70" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca70b18c31fe4e278ac3420cde483c70" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43329 } } }
I20260929 10:37:15.516661 3549 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca70b18c31fe4e278ac3420cde483c70 [sys.catalog]: This master's current role is: LEADER
I20260929 10:37:15.516947 3548 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca70b18c31fe4e278ac3420cde483c70 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ca70b18c31fe4e278ac3420cde483c70" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca70b18c31fe4e278ac3420cde483c70" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43329 } } }
I20260929 10:37:15.517032 3548 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca70b18c31fe4e278ac3420cde483c70 [sys.catalog]: This master's current role is: LEADER
I20260929 10:37:15.517078 3551 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:37:15.520665 3551 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:37:15.524473 3540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:15.513744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39857 (local address 127.0.0.1:43329)
0929 10:37:15.513881 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:15.513888 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:15.513963 (+ 75us) server_negotiation.cc:407] Connection header received
0929 10:37:15.514036 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:15.514039 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:15.514108 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:15.514191 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:15.515240 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:15.516049 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:15.517414 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:15.517770 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:15.517861 (+ 91us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:15.520057 (+ 2196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:15.520061 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:15.520064 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:37:15.520089 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:15.521866 (+ 1777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:15.522503 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:15.522507 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:15.522509 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:15.522597 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:15.523032 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:15.523037 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:15.523038 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:15.523298 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:15.523455 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:15.524173 (+ 718us) server_negotiation.cc:299] Negotiation successful
0929 10:37:15.524301 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260929 10:37:15.527155 3551 catalog_manager.cc:1347] Generated new cluster ID: c45f348e358048d9aab25c05bdc8800f
I20260929 10:37:15.527222 3551 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:37:15.538923 3551 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:37:15.539884 3551 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:37:15.545207 3551 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ca70b18c31fe4e278ac3420cde483c70: Generated new TSK 0
I20260929 10:37:15.546133 3551 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 10:37:15.556357 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 3469
2026-09-29T10:37:15Z chronyd exiting
W20260929 10:37:15.565693 3454 connection.cc:535] client connection to 127.0.0.1:43329 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43329): Connection reset by peer (error 104)
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1420 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790677981592896-15689-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:37:15 dist-test-slave-vg89 krb5kdc[3568](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:37:15 dist-test-slave-vg89 krb5kdc[3568](info): set up 2 sockets
Sep 29 10:37:15 dist-test-slave-vg89 krb5kdc[3568](info): commencing operation
krb5kdc: starting...
W20260929 10:37:16.594996 15689 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Sep 29 10:37:16 dist-test-slave-vg89 krb5kdc[3568](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678236, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:37:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:37:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790677981592896-15689-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790677981592896-15689-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 10:37:16.721803 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790677981592896-15689-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790677981592896-15689-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790677981592896-15689-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44029
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790677981592896-15689-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44029
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:37:16.832118 3581 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:37:16.832451 3581 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:37:16.832542 3581 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:37:16.836643 3581 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:37:16.836751 3581 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:37:16.836791 3581 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:37:16.836850 3581 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:37:16.836913 3581 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:37:16.842507 3581 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790677981592896-15689-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790677981592896-15689-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44029
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790677981592896-15689-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44029
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3581
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790677981592896-15689-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:37:16.843681 3581 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:37:16.844775 3581 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:37:16.853659 3587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:37:16.853742 3581 server_base.cc:1034] running on GCE node
W20260929 10:37:16.853754 3589 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:16.853672 3586 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:37:16.854588 3581 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:37:16.855710 3581 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:37:16.856913 3581 hybrid_clock.cc:648] HybridClock initialized: now 1790678236856897 us; error 28 us; skew 500 ppm
Sep 29 10:37:16 dist-test-slave-vg89 krb5kdc[3568](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678236, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:37:16.859844 3581 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 10:37:16.860744 3581 webserver.cc:460] Webserver started at http://127.0.0.1:36405/ using document root <none> and password file <none>
I20260929 10:37:16.861325 3581 fs_manager.cc:359] Metadata directory not provided
I20260929 10:37:16.861418 3581 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:37:16.861663 3581 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:37:16.863854 3581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790677981592896-15689-0/minicluster-data/master-0/data/instance:
uuid: "72dde21fafd346c3bec71a19facc32ce"
format_stamp: "Formatted at 2026-09-29 10:37:16 on dist-test-slave-vg89"
server_key: "645b14b666756bac78f691b879f98e04"
server_key_iv: "aadec70f6e4745490b0837679a984b3f"
server_key_version: "encryptionkey@0"
I20260929 10:37:16.864439 3581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790677981592896-15689-0/minicluster-data/master-0/wal/instance:
uuid: "72dde21fafd346c3bec71a19facc32ce"
format_stamp: "Formatted at 2026-09-29 10:37:16 on dist-test-slave-vg89"
server_key: "645b14b666756bac78f691b879f98e04"
server_key_iv: "aadec70f6e4745490b0837679a984b3f"
server_key_version: "encryptionkey@0"
I20260929 10:37:16.868286 3581 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 10:37:16.870836 3596 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:16.872118 3581 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 10:37:16.872264 3581 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790677981592896-15689-0/minicluster-data/master-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790677981592896-15689-0/minicluster-data/master-0/wal
uuid: "72dde21fafd346c3bec71a19facc32ce"
format_stamp: "Formatted at 2026-09-29 10:37:16 on dist-test-slave-vg89"
server_key: "645b14b666756bac78f691b879f98e04"
server_key_iv: "aadec70f6e4745490b0837679a984b3f"
server_key_version: "encryptionkey@0"
I20260929 10:37:16.872386 3581 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790677981592896-15689-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790677981592896-15689-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790677981592896-15689-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:37:16.892381 3581 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:37:16.895993 3581 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:37:16.896211 3581 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:37:16.903524 3581 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44029
I20260929 10:37:16.903551 3648 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44029 every 8 connection(s)
I20260929 10:37:16.904505 3581 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
I20260929 10:37:16.907364 3649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:37:16.908740 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 3581
I20260929 10:37:16.908902 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790677981592896-15689-0/minicluster-data/master-0/wal/instance
I20260929 10:37:16.909221 15689 external_mini_cluster.cc:1428] Setting key 4e713e9c4c5f418652dcbb9253d3a42e
I20260929 10:37:16.914018 3649 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72dde21fafd346c3bec71a19facc32ce: Bootstrap starting.
I20260929 10:37:16.916802 3649 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 72dde21fafd346c3bec71a19facc32ce: Neither blocks nor log segments found. Creating new log.
I20260929 10:37:16.917677 3649 log.cc:824] T 00000000000000000000000000000000 P 72dde21fafd346c3bec71a19facc32ce: Log is configured to *not* fsync() on all Append() calls
I20260929 10:37:16.918556 3652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:16.910734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58359 (local address 127.0.0.1:44029)
0929 10:37:16.911177 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:16.911192 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:16.911228 (+ 36us) server_negotiation.cc:407] Connection header received
0929 10:37:16.911869 (+ 641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:16.911886 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:16.912229 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:16.912523 (+ 294us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:16.912859 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:16.913828 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:16.915105 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:16.915594 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:16.915715 (+ 121us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:16.916644 (+ 929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:16.916649 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:16.916662 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:37:16.917114 (+ 452us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:16.917169 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:37:16.917192 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:16.917293 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:16.917509 (+ 216us) server_negotiation.cc:299] Negotiation successful
0929 10:37:16.917674 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":130,"threads_started":1}
I20260929 10:37:16.919750 3649 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72dde21fafd346c3bec71a19facc32ce: No bootstrap required, opened a new log
I20260929 10:37:16.921734 15689 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 10:37:16.924445 3649 raft_consensus.cc:348] T 00000000000000000000000000000000 P 72dde21fafd346c3bec71a19facc32ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72dde21fafd346c3bec71a19facc32ce" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44029 } }
I20260929 10:37:16.924750 3649 raft_consensus.cc:374] T 00000000000000000000000000000000 P 72dde21fafd346c3bec71a19facc32ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:37:16.924844 3649 raft_consensus.cc:729] T 00000000000000000000000000000000 P 72dde21fafd346c3bec71a19facc32ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72dde21fafd346c3bec71a19facc32ce, State: Initialized, Role: FOLLOWER
I20260929 10:37:16.925323 3649 consensus_queue.cc:260] T 00000000000000000000000000000000 P 72dde21fafd346c3bec71a19facc32ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72dde21fafd346c3bec71a19facc32ce" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44029 } }
I20260929 10:37:16.925450 3649 raft_consensus.cc:388] T 00000000000000000000000000000000 P 72dde21fafd346c3bec71a19facc32ce [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:37:16.925536 3649 raft_consensus.cc:482] T 00000000000000000000000000000000 P 72dde21fafd346c3bec71a19facc32ce [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:37:16.925639 3649 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 72dde21fafd346c3bec71a19facc32ce [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:37:16.926846 3649 raft_consensus.cc:504] T 00000000000000000000000000000000 P 72dde21fafd346c3bec71a19facc32ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72dde21fafd346c3bec71a19facc32ce" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44029 } }
I20260929 10:37:16.927222 3649 leader_election.cc:302] T 00000000000000000000000000000000 P 72dde21fafd346c3bec71a19facc32ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 72dde21fafd346c3bec71a19facc32ce; no voters:
I20260929 10:37:16.927556 3649 leader_election.cc:288] T 00000000000000000000000000000000 P 72dde21fafd346c3bec71a19facc32ce [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:37:16.927683 3659 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 72dde21fafd346c3bec71a19facc32ce [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:37:16.927973 3659 raft_consensus.cc:686] T 00000000000000000000000000000000 P 72dde21fafd346c3bec71a19facc32ce [term 1 LEADER]: Becoming Leader. State: Replica: 72dde21fafd346c3bec71a19facc32ce, State: Running, Role: LEADER
I20260929 10:37:16.928391 3659 consensus_queue.cc:237] T 00000000000000000000000000000000 P 72dde21fafd346c3bec71a19facc32ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72dde21fafd346c3bec71a19facc32ce" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44029 } }
I20260929 10:37:16.928948 3649 sys_catalog.cc:564] T 00000000000000000000000000000000 P 72dde21fafd346c3bec71a19facc32ce [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:37:16.928979 3652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:16.924150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44091 (local address 127.0.0.1:44029)
0929 10:37:16.924331 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:16.924334 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:16.924499 (+ 165us) server_negotiation.cc:407] Connection header received
0929 10:37:16.924593 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:16.924595 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:16.924633 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:16.924714 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:16.925122 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:16.925828 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:16.927008 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:16.927267 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:16.927398 (+ 131us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:16.928334 (+ 936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:16.928337 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:16.928340 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:37:16.928353 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:16.928381 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:37:16.928394 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:16.928487 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:16.928820 (+ 333us) server_negotiation.cc:299] Negotiation successful
0929 10:37:16.928856 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260929 10:37:16.930655 3660 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72dde21fafd346c3bec71a19facc32ce [sys.catalog]: SysCatalogTable state changed. Reason: New leader 72dde21fafd346c3bec71a19facc32ce. Latest consensus state: current_term: 1 leader_uuid: "72dde21fafd346c3bec71a19facc32ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72dde21fafd346c3bec71a19facc32ce" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44029 } } }
I20260929 10:37:16.930788 3660 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72dde21fafd346c3bec71a19facc32ce [sys.catalog]: This master's current role is: LEADER
I20260929 10:37:16.931077 3661 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72dde21fafd346c3bec71a19facc32ce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "72dde21fafd346c3bec71a19facc32ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72dde21fafd346c3bec71a19facc32ce" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44029 } } }
I20260929 10:37:16.931236 3661 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72dde21fafd346c3bec71a19facc32ce [sys.catalog]: This master's current role is: LEADER
W20260929 10:37:16.933990 3674 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 72dde21fafd346c3bec71a19facc32ce: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 10:37:16.934077 3674 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 10:37:16.934160 3672 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:37:16.934944 3672 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:37:16.940635 3672 catalog_manager.cc:1347] Generated new cluster ID: 5a041f7eb89742a7945fad18ee356aae
I20260929 10:37:16.940698 3672 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:37:16.947374 3672 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:37:16.948616 3672 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:37:16.954025 3672 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 72dde21fafd346c3bec71a19facc32ce: Generated new TSK 0
I20260929 10:37:16.954879 3672 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 10:37:16.964978 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 3581
W20260929 10:37:16.971172 3566 connection.cc:535] client connection to 127.0.0.1:44029 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44029): Connection reset by peer (error 104)
2026-09-29T10:37:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1406 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790677981592896-15689-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:37:17 dist-test-slave-vg89 krb5kdc[3680](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:37:17 dist-test-slave-vg89 krb5kdc[3680](info): set up 2 sockets
Sep 29 10:37:17 dist-test-slave-vg89 krb5kdc[3680](info): commencing operation
krb5kdc: starting...
W20260929 10:37:18.030963 15689 mini_kdc.cc:121] Time spent starting KDC: real 1.045s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Sep 29 10:37:18 dist-test-slave-vg89 krb5kdc[3680](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678238, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:37:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:37:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790677981592896-15689-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790677981592896-15689-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 10:37:18.157877 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790677981592896-15689-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790677981592896-15689-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790677981592896-15689-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37465
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790677981592896-15689-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37465
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:37:18.271922 3693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:37:18.272177 3693 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:37:18.272218 3693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:37:18.276306 3693 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:37:18.276360 3693 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:37:18.276379 3693 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:37:18.276396 3693 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:37:18.276410 3693 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:37:18.281620 3693 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790677981592896-15689-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790677981592896-15689-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37465
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790677981592896-15689-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37465
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3693
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790677981592896-15689-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:37:18.282665 3693 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:37:18.283740 3693 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:37:18.292222 3699 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:18.292255 3698 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:18.292222 3701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:37:18.292613 3693 server_base.cc:1034] running on GCE node
I20260929 10:37:18.293170 3693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:37:18.294212 3693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:37:18.295410 3693 hybrid_clock.cc:648] HybridClock initialized: now 1790678238295397 us; error 30 us; skew 500 ppm
Sep 29 10:37:18 dist-test-slave-vg89 krb5kdc[3680](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678238, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:37:18.298245 3693 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 10:37:18.299182 3693 webserver.cc:460] Webserver started at http://127.0.0.1:45235/ using document root <none> and password file <none>
I20260929 10:37:18.299788 3693 fs_manager.cc:359] Metadata directory not provided
I20260929 10:37:18.299883 3693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:37:18.300125 3693 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:37:18.302160 3693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790677981592896-15689-0/minicluster-data/master-0/data/instance:
uuid: "abce712bf6b74a7eb24dc054c3f65515"
format_stamp: "Formatted at 2026-09-29 10:37:18 on dist-test-slave-vg89"
server_key: "7c47739822208c5ab437e7eab9eb04e3"
server_key_iv: "d53fecd803fa0fe3f3099440a2e81198"
server_key_version: "encryptionkey@0"
I20260929 10:37:18.302767 3693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790677981592896-15689-0/minicluster-data/master-0/wal/instance:
uuid: "abce712bf6b74a7eb24dc054c3f65515"
format_stamp: "Formatted at 2026-09-29 10:37:18 on dist-test-slave-vg89"
server_key: "7c47739822208c5ab437e7eab9eb04e3"
server_key_iv: "d53fecd803fa0fe3f3099440a2e81198"
server_key_version: "encryptionkey@0"
I20260929 10:37:18.306481 3693 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260929 10:37:18.308903 3708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:18.310181 3693 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260929 10:37:18.310326 3693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790677981592896-15689-0/minicluster-data/master-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790677981592896-15689-0/minicluster-data/master-0/wal
uuid: "abce712bf6b74a7eb24dc054c3f65515"
format_stamp: "Formatted at 2026-09-29 10:37:18 on dist-test-slave-vg89"
server_key: "7c47739822208c5ab437e7eab9eb04e3"
server_key_iv: "d53fecd803fa0fe3f3099440a2e81198"
server_key_version: "encryptionkey@0"
I20260929 10:37:18.310515 3693 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790677981592896-15689-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790677981592896-15689-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790677981592896-15689-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:37:18.315196 3693 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:37:18.318481 3693 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:37:18.318670 3693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:37:18.325691 3693 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37465
I20260929 10:37:18.325716 3760 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37465 every 8 connection(s)
I20260929 10:37:18.326704 3693 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
I20260929 10:37:18.329437 3761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:37:18.334465 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 3693
I20260929 10:37:18.334612 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790677981592896-15689-0/minicluster-data/master-0/wal/instance
I20260929 10:37:18.334951 15689 external_mini_cluster.cc:1428] Setting key 566d59b2080aa6709e1dcdc093c12ec9
I20260929 10:37:18.335377 3761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abce712bf6b74a7eb24dc054c3f65515: Bootstrap starting.
I20260929 10:37:18.337879 3761 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P abce712bf6b74a7eb24dc054c3f65515: Neither blocks nor log segments found. Creating new log.
I20260929 10:37:18.338920 3761 log.cc:824] T 00000000000000000000000000000000 P abce712bf6b74a7eb24dc054c3f65515: Log is configured to *not* fsync() on all Append() calls
I20260929 10:37:18.340387 3764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:18.336358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54173 (local address 127.0.0.1:37465)
0929 10:37:18.336836 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:18.336851 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:18.336886 (+ 35us) server_negotiation.cc:407] Connection header received
0929 10:37:18.337527 (+ 641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:18.337544 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:18.337831 (+ 287us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:18.338182 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:18.339111 (+ 929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:18.339115 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:18.339127 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:37:18.339137 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:18.339197 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:37:18.339216 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:18.339355 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:18.339482 (+ 127us) server_negotiation.cc:299] Negotiation successful
0929 10:37:18.339641 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":145,"threads_started":1}
I20260929 10:37:18.341005 3761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abce712bf6b74a7eb24dc054c3f65515: No bootstrap required, opened a new log
I20260929 10:37:18.343425 15689 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 10:37:18.345746 3761 raft_consensus.cc:348] T 00000000000000000000000000000000 P abce712bf6b74a7eb24dc054c3f65515 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abce712bf6b74a7eb24dc054c3f65515" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37465 } }
I20260929 10:37:18.346128 3761 raft_consensus.cc:374] T 00000000000000000000000000000000 P abce712bf6b74a7eb24dc054c3f65515 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:37:18.346208 3761 raft_consensus.cc:729] T 00000000000000000000000000000000 P abce712bf6b74a7eb24dc054c3f65515 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abce712bf6b74a7eb24dc054c3f65515, State: Initialized, Role: FOLLOWER
I20260929 10:37:18.346741 3761 consensus_queue.cc:260] T 00000000000000000000000000000000 P abce712bf6b74a7eb24dc054c3f65515 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abce712bf6b74a7eb24dc054c3f65515" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37465 } }
I20260929 10:37:18.346896 3761 raft_consensus.cc:388] T 00000000000000000000000000000000 P abce712bf6b74a7eb24dc054c3f65515 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:37:18.346973 3761 raft_consensus.cc:482] T 00000000000000000000000000000000 P abce712bf6b74a7eb24dc054c3f65515 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:37:18.347096 3761 raft_consensus.cc:3037] T 00000000000000000000000000000000 P abce712bf6b74a7eb24dc054c3f65515 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:37:18.347748 3764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:18.345910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50881 (local address 127.0.0.1:37465)
0929 10:37:18.346045 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:18.346049 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:18.346250 (+ 201us) server_negotiation.cc:407] Connection header received
0929 10:37:18.346382 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:18.346384 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:18.346455 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:18.346540 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:18.347353 (+ 813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:18.347356 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:18.347358 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:37:18.347369 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:18.347398 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:37:18.347416 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:18.347502 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:18.347588 (+ 86us) server_negotiation.cc:299] Negotiation successful
0929 10:37:18.347628 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260929 10:37:18.348142 3761 raft_consensus.cc:504] T 00000000000000000000000000000000 P abce712bf6b74a7eb24dc054c3f65515 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abce712bf6b74a7eb24dc054c3f65515" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37465 } }
I20260929 10:37:18.348531 3761 leader_election.cc:302] T 00000000000000000000000000000000 P abce712bf6b74a7eb24dc054c3f65515 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: abce712bf6b74a7eb24dc054c3f65515; no voters:
I20260929 10:37:18.348883 3761 leader_election.cc:288] T 00000000000000000000000000000000 P abce712bf6b74a7eb24dc054c3f65515 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:37:18.348949 3771 raft_consensus.cc:2781] T 00000000000000000000000000000000 P abce712bf6b74a7eb24dc054c3f65515 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:37:18.349184 3771 raft_consensus.cc:686] T 00000000000000000000000000000000 P abce712bf6b74a7eb24dc054c3f65515 [term 1 LEADER]: Becoming Leader. State: Replica: abce712bf6b74a7eb24dc054c3f65515, State: Running, Role: LEADER
I20260929 10:37:18.349509 3771 consensus_queue.cc:237] T 00000000000000000000000000000000 P abce712bf6b74a7eb24dc054c3f65515 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abce712bf6b74a7eb24dc054c3f65515" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37465 } }
I20260929 10:37:18.350251 3761 sys_catalog.cc:564] T 00000000000000000000000000000000 P abce712bf6b74a7eb24dc054c3f65515 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:37:18.351362 3772 sys_catalog.cc:455] T 00000000000000000000000000000000 P abce712bf6b74a7eb24dc054c3f65515 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "abce712bf6b74a7eb24dc054c3f65515" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abce712bf6b74a7eb24dc054c3f65515" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37465 } } }
I20260929 10:37:18.351403 3773 sys_catalog.cc:455] T 00000000000000000000000000000000 P abce712bf6b74a7eb24dc054c3f65515 [sys.catalog]: SysCatalogTable state changed. Reason: New leader abce712bf6b74a7eb24dc054c3f65515. Latest consensus state: current_term: 1 leader_uuid: "abce712bf6b74a7eb24dc054c3f65515" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abce712bf6b74a7eb24dc054c3f65515" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37465 } } }
I20260929 10:37:18.351488 3772 sys_catalog.cc:458] T 00000000000000000000000000000000 P abce712bf6b74a7eb24dc054c3f65515 [sys.catalog]: This master's current role is: LEADER
I20260929 10:37:18.351504 3773 sys_catalog.cc:458] T 00000000000000000000000000000000 P abce712bf6b74a7eb24dc054c3f65515 [sys.catalog]: This master's current role is: LEADER
I20260929 10:37:18.351869 3780 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:37:18.354897 3780 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:37:18.360396 3780 catalog_manager.cc:1347] Generated new cluster ID: 4d5d5544ef3646e196e972ae0a4711de
I20260929 10:37:18.360455 3780 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:37:18.371073 3780 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:37:18.371452 3780 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:37:18.382696 3780 catalog_manager.cc:5868] T 00000000000000000000000000000000 P abce712bf6b74a7eb24dc054c3f65515: Generated new TSK 0
I20260929 10:37:18.383596 3780 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 10:37:18.390334 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 3693
2026-09-29T10:37:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1424 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790677981592896-15689-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:37:18 dist-test-slave-vg89 krb5kdc[3792](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:37:18 dist-test-slave-vg89 krb5kdc[3792](info): set up 2 sockets
Sep 29 10:37:18 dist-test-slave-vg89 krb5kdc[3792](info): commencing operation
krb5kdc: starting...
W20260929 10:37:19.426914 15689 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Sep 29 10:37:19 dist-test-slave-vg89 krb5kdc[3792](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678239, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:37:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:37:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790677981592896-15689-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790677981592896-15689-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 10:37:19.553853 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790677981592896-15689-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790677981592896-15689-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790677981592896-15689-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40713
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790677981592896-15689-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40713
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:37:19.667124 3805 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:37:19.667465 3805 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:37:19.667580 3805 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:37:19.671608 3805 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:37:19.671689 3805 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:37:19.671725 3805 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:37:19.671779 3805 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:37:19.671819 3805 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:37:19.676985 3805 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790677981592896-15689-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790677981592896-15689-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40713
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790677981592896-15689-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40713
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3805
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790677981592896-15689-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:37:19.678042 3805 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:37:19.679111 3805 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:37:19.687598 3813 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:19.687690 3811 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:19.687637 3810 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:37:19.687968 3805 server_base.cc:1034] running on GCE node
I20260929 10:37:19.688561 3805 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:37:19.689677 3805 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:37:19.690862 3805 hybrid_clock.cc:648] HybridClock initialized: now 1790678239690839 us; error 38 us; skew 500 ppm
Sep 29 10:37:19 dist-test-slave-vg89 krb5kdc[3792](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678239, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:37:19.697536 3805 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 10:37:19.698482 3805 webserver.cc:460] Webserver started at http://127.0.0.1:36763/ using document root <none> and password file <none>
I20260929 10:37:19.699059 3805 fs_manager.cc:359] Metadata directory not provided
I20260929 10:37:19.699148 3805 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:37:19.699379 3805 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:37:19.701834 3805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790677981592896-15689-0/minicluster-data/master-0/data/instance:
uuid: "e260fc6ea9cc4bee8dc628182a75edcc"
format_stamp: "Formatted at 2026-09-29 10:37:19 on dist-test-slave-vg89"
server_key: "b486bd6e38a72742eb5645835f606741"
server_key_iv: "f0c165ac360a5f1f93260dbb65f1fad5"
server_key_version: "encryptionkey@0"
I20260929 10:37:19.702459 3805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790677981592896-15689-0/minicluster-data/master-0/wal/instance:
uuid: "e260fc6ea9cc4bee8dc628182a75edcc"
format_stamp: "Formatted at 2026-09-29 10:37:19 on dist-test-slave-vg89"
server_key: "b486bd6e38a72742eb5645835f606741"
server_key_iv: "f0c165ac360a5f1f93260dbb65f1fad5"
server_key_version: "encryptionkey@0"
I20260929 10:37:19.706621 3805 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260929 10:37:19.709102 3820 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:19.710465 3805 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 10:37:19.710600 3805 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790677981592896-15689-0/minicluster-data/master-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790677981592896-15689-0/minicluster-data/master-0/wal
uuid: "e260fc6ea9cc4bee8dc628182a75edcc"
format_stamp: "Formatted at 2026-09-29 10:37:19 on dist-test-slave-vg89"
server_key: "b486bd6e38a72742eb5645835f606741"
server_key_iv: "f0c165ac360a5f1f93260dbb65f1fad5"
server_key_version: "encryptionkey@0"
I20260929 10:37:19.710714 3805 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790677981592896-15689-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790677981592896-15689-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790677981592896-15689-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:37:19.725689 3805 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:37:19.729075 3805 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:37:19.729300 3805 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:37:19.736518 3805 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40713
I20260929 10:37:19.736544 3872 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40713 every 8 connection(s)
I20260929 10:37:19.737542 3805 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
I20260929 10:37:19.740327 3873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:37:19.740813 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 3805
I20260929 10:37:19.740938 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790677981592896-15689-0/minicluster-data/master-0/wal/instance
I20260929 10:37:19.741238 15689 external_mini_cluster.cc:1428] Setting key 9eac9744128d0d68c17c6fa9754a4d6b
I20260929 10:37:19.747548 3873 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e260fc6ea9cc4bee8dc628182a75edcc: Bootstrap starting.
I20260929 10:37:19.749981 3873 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e260fc6ea9cc4bee8dc628182a75edcc: Neither blocks nor log segments found. Creating new log.
Sep 29 10:37:19 dist-test-slave-vg89 krb5kdc[3792](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678239, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260929 10:37:19.750815 3873 log.cc:824] T 00000000000000000000000000000000 P e260fc6ea9cc4bee8dc628182a75edcc: Log is configured to *not* fsync() on all Append() calls
I20260929 10:37:19.753032 3873 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e260fc6ea9cc4bee8dc628182a75edcc: No bootstrap required, opened a new log
I20260929 10:37:19.756042 3876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:19.742696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46565 (local address 127.0.0.1:40713)
0929 10:37:19.743130 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:19.743149 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:19.743194 (+ 45us) server_negotiation.cc:407] Connection header received
0929 10:37:19.743943 (+ 749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:19.743967 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:19.744406 (+ 439us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:19.744764 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:19.745445 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:19.746989 (+ 1544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:19.747876 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:19.748529 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:19.748692 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:19.751213 (+ 2521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:19.751223 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:19.751237 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:37:19.751260 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:19.753102 (+ 1842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:19.753514 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:19.753520 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:19.753525 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:19.753659 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:19.753907 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:19.753910 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:19.753912 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:19.754379 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:19.754576 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:19.754767 (+ 191us) server_negotiation.cc:299] Negotiation successful
0929 10:37:19.755094 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":111,"threads_started":1}
I20260929 10:37:19.757460 3873 raft_consensus.cc:348] T 00000000000000000000000000000000 P e260fc6ea9cc4bee8dc628182a75edcc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e260fc6ea9cc4bee8dc628182a75edcc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40713 } }
I20260929 10:37:19.757700 3873 raft_consensus.cc:374] T 00000000000000000000000000000000 P e260fc6ea9cc4bee8dc628182a75edcc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:37:19.757759 3873 raft_consensus.cc:729] T 00000000000000000000000000000000 P e260fc6ea9cc4bee8dc628182a75edcc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e260fc6ea9cc4bee8dc628182a75edcc, State: Initialized, Role: FOLLOWER
I20260929 10:37:19.758296 3873 consensus_queue.cc:260] T 00000000000000000000000000000000 P e260fc6ea9cc4bee8dc628182a75edcc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e260fc6ea9cc4bee8dc628182a75edcc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40713 } }
I20260929 10:37:19.758486 3873 raft_consensus.cc:388] T 00000000000000000000000000000000 P e260fc6ea9cc4bee8dc628182a75edcc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:37:19.758586 3873 raft_consensus.cc:482] T 00000000000000000000000000000000 P e260fc6ea9cc4bee8dc628182a75edcc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:37:19.758688 3873 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e260fc6ea9cc4bee8dc628182a75edcc [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:37:19.759045 15689 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 10:37:19.759831 3873 raft_consensus.cc:504] T 00000000000000000000000000000000 P e260fc6ea9cc4bee8dc628182a75edcc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e260fc6ea9cc4bee8dc628182a75edcc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40713 } }
I20260929 10:37:19.760200 3873 leader_election.cc:302] T 00000000000000000000000000000000 P e260fc6ea9cc4bee8dc628182a75edcc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e260fc6ea9cc4bee8dc628182a75edcc; no voters:
I20260929 10:37:19.760553 3873 leader_election.cc:288] T 00000000000000000000000000000000 P e260fc6ea9cc4bee8dc628182a75edcc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:37:19.760646 3881 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e260fc6ea9cc4bee8dc628182a75edcc [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:37:19.760885 3881 raft_consensus.cc:686] T 00000000000000000000000000000000 P e260fc6ea9cc4bee8dc628182a75edcc [term 1 LEADER]: Becoming Leader. State: Replica: e260fc6ea9cc4bee8dc628182a75edcc, State: Running, Role: LEADER
I20260929 10:37:19.761251 3881 consensus_queue.cc:237] T 00000000000000000000000000000000 P e260fc6ea9cc4bee8dc628182a75edcc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e260fc6ea9cc4bee8dc628182a75edcc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40713 } }
I20260929 10:37:19.761811 3873 sys_catalog.cc:564] T 00000000000000000000000000000000 P e260fc6ea9cc4bee8dc628182a75edcc [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:37:19.763067 3884 sys_catalog.cc:455] T 00000000000000000000000000000000 P e260fc6ea9cc4bee8dc628182a75edcc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e260fc6ea9cc4bee8dc628182a75edcc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e260fc6ea9cc4bee8dc628182a75edcc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40713 } } }
I20260929 10:37:19.763190 3884 sys_catalog.cc:458] T 00000000000000000000000000000000 P e260fc6ea9cc4bee8dc628182a75edcc [sys.catalog]: This master's current role is: LEADER
I20260929 10:37:19.763407 3884 sys_catalog.cc:455] T 00000000000000000000000000000000 P e260fc6ea9cc4bee8dc628182a75edcc [sys.catalog]: SysCatalogTable state changed. Reason: New leader e260fc6ea9cc4bee8dc628182a75edcc. Latest consensus state: current_term: 1 leader_uuid: "e260fc6ea9cc4bee8dc628182a75edcc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e260fc6ea9cc4bee8dc628182a75edcc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40713 } } }
I20260929 10:37:19.763474 3884 sys_catalog.cc:458] T 00000000000000000000000000000000 P e260fc6ea9cc4bee8dc628182a75edcc [sys.catalog]: This master's current role is: LEADER
I20260929 10:37:19.764412 3892 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:37:19.767464 3892 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:37:19.769815 3876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:19.761314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39761 (local address 127.0.0.1:40713)
0929 10:37:19.761410 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:19.761414 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:19.761815 (+ 401us) server_negotiation.cc:407] Connection header received
0929 10:37:19.761991 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:19.761994 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:19.762038 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:19.762130 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:19.763042 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:19.763832 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:19.764938 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:19.765250 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:19.765361 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:19.766705 (+ 1344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:19.766708 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:19.766711 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:37:19.766735 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:19.768168 (+ 1433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:19.768592 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:19.768596 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:19.768598 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:19.768676 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:19.768949 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:19.768953 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:19.768954 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:19.769126 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:19.769253 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:19.769521 (+ 268us) server_negotiation.cc:299] Negotiation successful
0929 10:37:19.769640 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":37}
I20260929 10:37:19.773059 3892 catalog_manager.cc:1347] Generated new cluster ID: 675820ca91d544ec91f83718f85cb89c
I20260929 10:37:19.773162 3892 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:37:19.780646 3892 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:37:19.781558 3892 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:37:19.794060 3892 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e260fc6ea9cc4bee8dc628182a75edcc: Generated new TSK 0
I20260929 10:37:19.794926 3892 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 10:37:19.799278 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 3805
2026-09-29T10:37:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1409 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790677981592896-15689-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:37:19 dist-test-slave-vg89 krb5kdc[3904](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:37:19 dist-test-slave-vg89 krb5kdc[3904](info): set up 2 sockets
Sep 29 10:37:19 dist-test-slave-vg89 krb5kdc[3904](info): commencing operation
krb5kdc: starting...
W20260929 10:37:20.899013 15689 mini_kdc.cc:121] Time spent starting KDC: real 1.079s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Sep 29 10:37:20 dist-test-slave-vg89 krb5kdc[3904](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678240, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:37:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:37:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790677981592896-15689-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790677981592896-15689-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 10:37:21.023491 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790677981592896-15689-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790677981592896-15689-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790677981592896-15689-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35369
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790677981592896-15689-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35369
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:37:21.135149 3917 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:37:21.135417 3917 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:37:21.135462 3917 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:37:21.139617 3917 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:37:21.139672 3917 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:37:21.139691 3917 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:37:21.139707 3917 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:37:21.139755 3917 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:37:21.145049 3917 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790677981592896-15689-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790677981592896-15689-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35369
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790677981592896-15689-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35369
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3917
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790677981592896-15689-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:37:21.146092 3917 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:37:21.147130 3917 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:37:21.155558 3925 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:21.155557 3923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:21.155665 3922 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:37:21.156067 3917 server_base.cc:1034] running on GCE node
I20260929 10:37:21.156631 3917 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:37:21.157694 3917 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:37:21.158874 3917 hybrid_clock.cc:648] HybridClock initialized: now 1790678241158849 us; error 43 us; skew 500 ppm
Sep 29 10:37:21 dist-test-slave-vg89 krb5kdc[3904](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678241, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:37:21.161581 3917 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 10:37:21.162536 3917 webserver.cc:460] Webserver started at http://127.0.0.1:46201/ using document root <none> and password file <none>
I20260929 10:37:21.163113 3917 fs_manager.cc:359] Metadata directory not provided
I20260929 10:37:21.163208 3917 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:37:21.163439 3917 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:37:21.165424 3917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790677981592896-15689-0/minicluster-data/master-0/data/instance:
uuid: "f5ace5fb233b4b29a2a6217787da290e"
format_stamp: "Formatted at 2026-09-29 10:37:21 on dist-test-slave-vg89"
server_key: "999b06d0260f06f5f9e989d38c6d6ff2"
server_key_iv: "483ac5e8f37e5d934fd728e719e8b6d2"
server_key_version: "encryptionkey@0"
I20260929 10:37:21.165994 3917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790677981592896-15689-0/minicluster-data/master-0/wal/instance:
uuid: "f5ace5fb233b4b29a2a6217787da290e"
format_stamp: "Formatted at 2026-09-29 10:37:21 on dist-test-slave-vg89"
server_key: "999b06d0260f06f5f9e989d38c6d6ff2"
server_key_iv: "483ac5e8f37e5d934fd728e719e8b6d2"
server_key_version: "encryptionkey@0"
I20260929 10:37:21.169633 3917 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260929 10:37:21.172228 3932 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:21.173460 3917 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260929 10:37:21.173600 3917 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790677981592896-15689-0/minicluster-data/master-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790677981592896-15689-0/minicluster-data/master-0/wal
uuid: "f5ace5fb233b4b29a2a6217787da290e"
format_stamp: "Formatted at 2026-09-29 10:37:21 on dist-test-slave-vg89"
server_key: "999b06d0260f06f5f9e989d38c6d6ff2"
server_key_iv: "483ac5e8f37e5d934fd728e719e8b6d2"
server_key_version: "encryptionkey@0"
I20260929 10:37:21.173712 3917 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790677981592896-15689-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790677981592896-15689-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790677981592896-15689-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:37:21.201593 3917 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:37:21.205184 3917 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:37:21.205405 3917 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:37:21.212790 3917 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35369
I20260929 10:37:21.212809 3984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35369 every 8 connection(s)
I20260929 10:37:21.213752 3917 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
I20260929 10:37:21.216569 3985 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:37:21.220152 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 3917
I20260929 10:37:21.220319 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790677981592896-15689-0/minicluster-data/master-0/wal/instance
I20260929 10:37:21.220619 15689 external_mini_cluster.cc:1428] Setting key b3b12cfa0c252cdfd3c3a3f9a64745d8
I20260929 10:37:21.222631 3985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5ace5fb233b4b29a2a6217787da290e: Bootstrap starting.
I20260929 10:37:21.224737 3985 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f5ace5fb233b4b29a2a6217787da290e: Neither blocks nor log segments found. Creating new log.
I20260929 10:37:21.225454 3985 log.cc:824] T 00000000000000000000000000000000 P f5ace5fb233b4b29a2a6217787da290e: Log is configured to *not* fsync() on all Append() calls
I20260929 10:37:21.227289 3985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5ace5fb233b4b29a2a6217787da290e: No bootstrap required, opened a new log
I20260929 10:37:21.229848 3988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:21.221997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39105 (local address 127.0.0.1:35369)
0929 10:37:21.222468 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:21.222485 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:21.222521 (+ 36us) server_negotiation.cc:407] Connection header received
0929 10:37:21.223189 (+ 668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:21.223206 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:21.223557 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:21.223891 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:21.224305 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:21.225341 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:21.226289 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:21.226825 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:21.227001 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:21.227893 (+ 892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:21.227899 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:21.227913 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:37:21.228280 (+ 367us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:21.228331 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:37:21.228353 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:21.228471 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:21.228596 (+ 125us) server_negotiation.cc:299] Negotiation successful
0929 10:37:21.228796 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":122,"threads_started":1}
I20260929 10:37:21.230259 3985 raft_consensus.cc:348] T 00000000000000000000000000000000 P f5ace5fb233b4b29a2a6217787da290e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5ace5fb233b4b29a2a6217787da290e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35369 } }
I20260929 10:37:21.230540 3985 raft_consensus.cc:374] T 00000000000000000000000000000000 P f5ace5fb233b4b29a2a6217787da290e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:37:21.230604 3985 raft_consensus.cc:729] T 00000000000000000000000000000000 P f5ace5fb233b4b29a2a6217787da290e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ace5fb233b4b29a2a6217787da290e, State: Initialized, Role: FOLLOWER
I20260929 10:37:21.231110 3985 consensus_queue.cc:260] T 00000000000000000000000000000000 P f5ace5fb233b4b29a2a6217787da290e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5ace5fb233b4b29a2a6217787da290e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35369 } }
I20260929 10:37:21.231263 3985 raft_consensus.cc:388] T 00000000000000000000000000000000 P f5ace5fb233b4b29a2a6217787da290e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:37:21.231328 3985 raft_consensus.cc:482] T 00000000000000000000000000000000 P f5ace5fb233b4b29a2a6217787da290e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:37:21.231417 3985 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f5ace5fb233b4b29a2a6217787da290e [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:37:21.232455 3985 raft_consensus.cc:504] T 00000000000000000000000000000000 P f5ace5fb233b4b29a2a6217787da290e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5ace5fb233b4b29a2a6217787da290e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35369 } }
I20260929 10:37:21.232755 15689 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 10:37:21.232812 3985 leader_election.cc:302] T 00000000000000000000000000000000 P f5ace5fb233b4b29a2a6217787da290e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f5ace5fb233b4b29a2a6217787da290e; no voters:
I20260929 10:37:21.233167 3985 leader_election.cc:288] T 00000000000000000000000000000000 P f5ace5fb233b4b29a2a6217787da290e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:37:21.233238 3990 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f5ace5fb233b4b29a2a6217787da290e [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:37:21.233455 3990 raft_consensus.cc:686] T 00000000000000000000000000000000 P f5ace5fb233b4b29a2a6217787da290e [term 1 LEADER]: Becoming Leader. State: Replica: f5ace5fb233b4b29a2a6217787da290e, State: Running, Role: LEADER
I20260929 10:37:21.233748 3990 consensus_queue.cc:237] T 00000000000000000000000000000000 P f5ace5fb233b4b29a2a6217787da290e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5ace5fb233b4b29a2a6217787da290e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35369 } }
I20260929 10:37:21.234443 3985 sys_catalog.cc:564] T 00000000000000000000000000000000 P f5ace5fb233b4b29a2a6217787da290e [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:37:21.235385 3991 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5ace5fb233b4b29a2a6217787da290e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f5ace5fb233b4b29a2a6217787da290e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5ace5fb233b4b29a2a6217787da290e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35369 } } }
I20260929 10:37:21.235525 3991 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5ace5fb233b4b29a2a6217787da290e [sys.catalog]: This master's current role is: LEADER
I20260929 10:37:21.235924 4004 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:37:21.235877 3992 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5ace5fb233b4b29a2a6217787da290e [sys.catalog]: SysCatalogTable state changed. Reason: New leader f5ace5fb233b4b29a2a6217787da290e. Latest consensus state: current_term: 1 leader_uuid: "f5ace5fb233b4b29a2a6217787da290e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5ace5fb233b4b29a2a6217787da290e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35369 } } }
I20260929 10:37:21.236029 3992 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5ace5fb233b4b29a2a6217787da290e [sys.catalog]: This master's current role is: LEADER
I20260929 10:37:21.238976 4004 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:37:21.243698 3988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:21.235025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46607 (local address 127.0.0.1:35369)
0929 10:37:21.235132 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:21.235138 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:21.237841 (+ 2703us) server_negotiation.cc:407] Connection header received
0929 10:37:21.238045 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:21.238048 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:21.238093 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:21.238167 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:21.238456 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:21.239048 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:21.241828 (+ 2780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:21.242169 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:21.242260 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:21.243291 (+ 1031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:21.243294 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:21.243297 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:37:21.243309 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:21.243340 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:37:21.243350 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:21.243430 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:21.243519 (+ 89us) server_negotiation.cc:299] Negotiation successful
0929 10:37:21.243567 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260929 10:37:21.248008 4004 catalog_manager.cc:1347] Generated new cluster ID: 79f0665d117c4621abdd75306f2fd533
I20260929 10:37:21.248091 4004 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:37:21.275032 4004 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:37:21.276310 4004 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:37:21.282948 4004 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f5ace5fb233b4b29a2a6217787da290e: Generated new TSK 0
I20260929 10:37:21.283588 4004 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 10:37:21.296044 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 3917
2026-09-29T10:37:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1497 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790677981592896-15689-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:37:21 dist-test-slave-vg89 krb5kdc[4016](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:37:21 dist-test-slave-vg89 krb5kdc[4016](info): set up 2 sockets
Sep 29 10:37:21 dist-test-slave-vg89 krb5kdc[4016](info): commencing operation
krb5kdc: starting...
W20260929 10:37:22.339002 15689 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Sep 29 10:37:22 dist-test-slave-vg89 krb5kdc[4016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678242, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:37:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:37:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790677981592896-15689-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790677981592896-15689-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 10:37:22.462630 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790677981592896-15689-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790677981592896-15689-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790677981592896-15689-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45717
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790677981592896-15689-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45717
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:37:22.574266 4029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:37:22.574554 4029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:37:22.574638 4029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:37:22.578732 4029 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:37:22.578789 4029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:37:22.578809 4029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:37:22.578825 4029 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:37:22.578840 4029 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:37:22.584116 4029 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790677981592896-15689-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790677981592896-15689-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45717
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790677981592896-15689-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45717
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4029
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790677981592896-15689-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:37:22.585201 4029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:37:22.586225 4029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:37:22.594712 4034 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:22.594717 4035 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:22.594679 4037 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:37:22.594898 4029 server_base.cc:1034] running on GCE node
I20260929 10:37:22.595402 4029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:37:22.596473 4029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:37:22.597661 4029 hybrid_clock.cc:648] HybridClock initialized: now 1790678242597628 us; error 42 us; skew 500 ppm
Sep 29 10:37:22 dist-test-slave-vg89 krb5kdc[4016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678242, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:37:22.600601 4029 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 10:37:22.601606 4029 webserver.cc:460] Webserver started at http://127.0.0.1:42661/ using document root <none> and password file <none>
I20260929 10:37:22.602206 4029 fs_manager.cc:359] Metadata directory not provided
I20260929 10:37:22.602310 4029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:37:22.602584 4029 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:37:22.604745 4029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790677981592896-15689-0/minicluster-data/master-0/data/instance:
uuid: "8865cc2f39f140f084640d6c6c53b17c"
format_stamp: "Formatted at 2026-09-29 10:37:22 on dist-test-slave-vg89"
server_key: "90391bcafa1dbb9dfaa8fbfe11c88ebb"
server_key_iv: "187d624049bc01ee297a236bdb9ac801"
server_key_version: "encryptionkey@0"
I20260929 10:37:22.605362 4029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790677981592896-15689-0/minicluster-data/master-0/wal/instance:
uuid: "8865cc2f39f140f084640d6c6c53b17c"
format_stamp: "Formatted at 2026-09-29 10:37:22 on dist-test-slave-vg89"
server_key: "90391bcafa1dbb9dfaa8fbfe11c88ebb"
server_key_iv: "187d624049bc01ee297a236bdb9ac801"
server_key_version: "encryptionkey@0"
I20260929 10:37:22.609395 4029 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260929 10:37:22.612104 4044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:22.613401 4029 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260929 10:37:22.613550 4029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790677981592896-15689-0/minicluster-data/master-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790677981592896-15689-0/minicluster-data/master-0/wal
uuid: "8865cc2f39f140f084640d6c6c53b17c"
format_stamp: "Formatted at 2026-09-29 10:37:22 on dist-test-slave-vg89"
server_key: "90391bcafa1dbb9dfaa8fbfe11c88ebb"
server_key_iv: "187d624049bc01ee297a236bdb9ac801"
server_key_version: "encryptionkey@0"
I20260929 10:37:22.613667 4029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790677981592896-15689-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790677981592896-15689-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790677981592896-15689-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:37:22.618916 4029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:37:22.622123 4029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:37:22.622339 4029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:37:22.629989 4029 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45717
I20260929 10:37:22.630015 4096 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45717 every 8 connection(s)
I20260929 10:37:22.631049 4029 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
I20260929 10:37:22.633708 4097 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:37:22.638875 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 4029
I20260929 10:37:22.638995 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790677981592896-15689-0/minicluster-data/master-0/wal/instance
I20260929 10:37:22.639310 15689 external_mini_cluster.cc:1428] Setting key ba1331e0d03791b7d082d1d43be2a491
I20260929 10:37:22.639389 4097 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8865cc2f39f140f084640d6c6c53b17c: Bootstrap starting.
I20260929 10:37:22.641918 4097 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8865cc2f39f140f084640d6c6c53b17c: Neither blocks nor log segments found. Creating new log.
I20260929 10:37:22.642733 4097 log.cc:824] T 00000000000000000000000000000000 P 8865cc2f39f140f084640d6c6c53b17c: Log is configured to *not* fsync() on all Append() calls
I20260929 10:37:22.644776 4100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:22.640738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33725 (local address 127.0.0.1:45717)
0929 10:37:22.641201 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:22.641216 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:22.641251 (+ 35us) server_negotiation.cc:407] Connection header received
0929 10:37:22.641896 (+ 645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:22.641913 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:22.642253 (+ 340us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:22.642656 (+ 403us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:22.643484 (+ 828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:22.643493 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:22.643510 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:37:22.643522 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:22.643600 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:37:22.643624 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:22.643746 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:22.643898 (+ 152us) server_negotiation.cc:299] Negotiation successful
0929 10:37:22.644061 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":132,"threads_started":1}
I20260929 10:37:22.644814 4097 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8865cc2f39f140f084640d6c6c53b17c: No bootstrap required, opened a new log
I20260929 10:37:22.647656 15689 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 10:37:22.648958 4097 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8865cc2f39f140f084640d6c6c53b17c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8865cc2f39f140f084640d6c6c53b17c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45717 } }
I20260929 10:37:22.649223 4097 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8865cc2f39f140f084640d6c6c53b17c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:37:22.649287 4097 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8865cc2f39f140f084640d6c6c53b17c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8865cc2f39f140f084640d6c6c53b17c, State: Initialized, Role: FOLLOWER
I20260929 10:37:22.649781 4097 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8865cc2f39f140f084640d6c6c53b17c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8865cc2f39f140f084640d6c6c53b17c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45717 } }
I20260929 10:37:22.649977 4097 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8865cc2f39f140f084640d6c6c53b17c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:37:22.650058 4097 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8865cc2f39f140f084640d6c6c53b17c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:37:22.650166 4097 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8865cc2f39f140f084640d6c6c53b17c [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:37:22.651428 4100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:22.649681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35215 (local address 127.0.0.1:45717)
0929 10:37:22.649830 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:22.649834 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:22.650071 (+ 237us) server_negotiation.cc:407] Connection header received
0929 10:37:22.650215 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:22.650218 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:22.650251 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:22.650342 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:22.651042 (+ 700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:22.651045 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:22.651047 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:37:22.651057 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:22.651080 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:37:22.651089 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:22.651157 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:22.651267 (+ 110us) server_negotiation.cc:299] Negotiation successful
0929 10:37:22.651299 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260929 10:37:22.651566 4097 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8865cc2f39f140f084640d6c6c53b17c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8865cc2f39f140f084640d6c6c53b17c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45717 } }
I20260929 10:37:22.651918 4097 leader_election.cc:302] T 00000000000000000000000000000000 P 8865cc2f39f140f084640d6c6c53b17c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8865cc2f39f140f084640d6c6c53b17c; no voters:
I20260929 10:37:22.652257 4097 leader_election.cc:288] T 00000000000000000000000000000000 P 8865cc2f39f140f084640d6c6c53b17c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:37:22.652347 4107 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8865cc2f39f140f084640d6c6c53b17c [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:37:22.652544 4107 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8865cc2f39f140f084640d6c6c53b17c [term 1 LEADER]: Becoming Leader. State: Replica: 8865cc2f39f140f084640d6c6c53b17c, State: Running, Role: LEADER
I20260929 10:37:22.652894 4107 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8865cc2f39f140f084640d6c6c53b17c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8865cc2f39f140f084640d6c6c53b17c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45717 } }
I20260929 10:37:22.653628 4097 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8865cc2f39f140f084640d6c6c53b17c [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:37:22.654650 4108 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8865cc2f39f140f084640d6c6c53b17c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8865cc2f39f140f084640d6c6c53b17c. Latest consensus state: current_term: 1 leader_uuid: "8865cc2f39f140f084640d6c6c53b17c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8865cc2f39f140f084640d6c6c53b17c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45717 } } }
I20260929 10:37:22.654798 4108 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8865cc2f39f140f084640d6c6c53b17c [sys.catalog]: This master's current role is: LEADER
I20260929 10:37:22.655074 4109 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8865cc2f39f140f084640d6c6c53b17c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8865cc2f39f140f084640d6c6c53b17c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8865cc2f39f140f084640d6c6c53b17c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45717 } } }
I20260929 10:37:22.655200 4109 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8865cc2f39f140f084640d6c6c53b17c [sys.catalog]: This master's current role is: LEADER
I20260929 10:37:22.655236 4116 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:37:22.658169 4116 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:37:22.663401 4116 catalog_manager.cc:1347] Generated new cluster ID: c36ea7ef9b484d62aa5a7df544926a64
I20260929 10:37:22.663477 4116 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:37:22.683815 4116 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:37:22.684166 4116 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:37:22.692333 4116 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8865cc2f39f140f084640d6c6c53b17c: Generated new TSK 0
I20260929 10:37:22.693060 4116 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 10:37:22.706240 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 4029
2026-09-29T10:37:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1409 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790677981592896-15689-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:37:22 dist-test-slave-vg89 krb5kdc[4128](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:37:22 dist-test-slave-vg89 krb5kdc[4128](info): set up 2 sockets
Sep 29 10:37:22 dist-test-slave-vg89 krb5kdc[4128](info): commencing operation
krb5kdc: starting...
W20260929 10:37:23.751130 15689 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Sep 29 10:37:23 dist-test-slave-vg89 krb5kdc[4128](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678243, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:37:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:37:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790677981592896-15689-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790677981592896-15689-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 10:37:23.877766 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790677981592896-15689-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790677981592896-15689-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790677981592896-15689-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38799
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790677981592896-15689-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38799
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:37:23.989387 4141 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:37:23.989679 4141 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:37:23.989723 4141 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:37:23.993925 4141 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:37:23.993995 4141 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:37:23.994015 4141 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:37:23.994033 4141 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:37:23.994076 4141 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:37:23.999532 4141 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790677981592896-15689-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790677981592896-15689-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38799
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790677981592896-15689-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38799
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4141
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790677981592896-15689-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:37:24.000656 4141 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:37:24.001741 4141 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:37:24.010692 4146 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:24.010694 4147 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:37:24.010849 4141 server_base.cc:1034] running on GCE node
W20260929 10:37:24.010797 4149 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:37:24.011641 4141 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:37:24.012712 4141 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:37:24.013890 4141 hybrid_clock.cc:648] HybridClock initialized: now 1790678244013880 us; error 31 us; skew 500 ppm
Sep 29 10:37:24 dist-test-slave-vg89 krb5kdc[4128](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678244, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:37:24.016765 4141 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 10:37:24.017632 4141 webserver.cc:460] Webserver started at http://127.0.0.1:37917/ using document root <none> and password file <none>
I20260929 10:37:24.018162 4141 fs_manager.cc:359] Metadata directory not provided
I20260929 10:37:24.018229 4141 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:37:24.018445 4141 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:37:24.020533 4141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790677981592896-15689-0/minicluster-data/master-0/data/instance:
uuid: "a51b0e15e75942928e3fe4b922b50dd1"
format_stamp: "Formatted at 2026-09-29 10:37:24 on dist-test-slave-vg89"
server_key: "7ec491e56ce6a1c667c93e2b0467298b"
server_key_iv: "57f0a8ddcbf0bd04b9fc6199e3bf984a"
server_key_version: "encryptionkey@0"
I20260929 10:37:24.021051 4141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790677981592896-15689-0/minicluster-data/master-0/wal/instance:
uuid: "a51b0e15e75942928e3fe4b922b50dd1"
format_stamp: "Formatted at 2026-09-29 10:37:24 on dist-test-slave-vg89"
server_key: "7ec491e56ce6a1c667c93e2b0467298b"
server_key_iv: "57f0a8ddcbf0bd04b9fc6199e3bf984a"
server_key_version: "encryptionkey@0"
I20260929 10:37:24.024945 4141 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 10:37:24.027578 4156 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:24.028967 4141 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 10:37:24.029075 4141 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790677981592896-15689-0/minicluster-data/master-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790677981592896-15689-0/minicluster-data/master-0/wal
uuid: "a51b0e15e75942928e3fe4b922b50dd1"
format_stamp: "Formatted at 2026-09-29 10:37:24 on dist-test-slave-vg89"
server_key: "7ec491e56ce6a1c667c93e2b0467298b"
server_key_iv: "57f0a8ddcbf0bd04b9fc6199e3bf984a"
server_key_version: "encryptionkey@0"
I20260929 10:37:24.029165 4141 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790677981592896-15689-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790677981592896-15689-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790677981592896-15689-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:37:24.043486 4141 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:37:24.046886 4141 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:37:24.047060 4141 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:37:24.054612 4141 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38799
I20260929 10:37:24.054647 4208 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38799 every 8 connection(s)
I20260929 10:37:24.055627 4141 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
I20260929 10:37:24.058631 4209 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:37:24.064713 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 4141
I20260929 10:37:24.064823 4209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a51b0e15e75942928e3fe4b922b50dd1: Bootstrap starting.
I20260929 10:37:24.064888 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790677981592896-15689-0/minicluster-data/master-0/wal/instance
I20260929 10:37:24.065199 15689 external_mini_cluster.cc:1428] Setting key 54eebbcf46cc8bec4de314012e4d03a1
I20260929 10:37:24.067564 4209 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a51b0e15e75942928e3fe4b922b50dd1: Neither blocks nor log segments found. Creating new log.
I20260929 10:37:24.068334 4209 log.cc:824] T 00000000000000000000000000000000 P a51b0e15e75942928e3fe4b922b50dd1: Log is configured to *not* fsync() on all Append() calls
I20260929 10:37:24.070302 4209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a51b0e15e75942928e3fe4b922b50dd1: No bootstrap required, opened a new log
I20260929 10:37:24.073992 4209 raft_consensus.cc:348] T 00000000000000000000000000000000 P a51b0e15e75942928e3fe4b922b50dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a51b0e15e75942928e3fe4b922b50dd1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38799 } }
I20260929 10:37:24.074218 4209 raft_consensus.cc:374] T 00000000000000000000000000000000 P a51b0e15e75942928e3fe4b922b50dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:37:24.074282 4209 raft_consensus.cc:729] T 00000000000000000000000000000000 P a51b0e15e75942928e3fe4b922b50dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a51b0e15e75942928e3fe4b922b50dd1, State: Initialized, Role: FOLLOWER
Sep 29 10:37:24 dist-test-slave-vg89 krb5kdc[4128](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678243, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260929 10:37:24.074788 4209 consensus_queue.cc:260] T 00000000000000000000000000000000 P a51b0e15e75942928e3fe4b922b50dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a51b0e15e75942928e3fe4b922b50dd1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38799 } }
I20260929 10:37:24.074997 4209 raft_consensus.cc:388] T 00000000000000000000000000000000 P a51b0e15e75942928e3fe4b922b50dd1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:37:24.075088 4209 raft_consensus.cc:482] T 00000000000000000000000000000000 P a51b0e15e75942928e3fe4b922b50dd1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:37:24.075210 4209 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a51b0e15e75942928e3fe4b922b50dd1 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:37:24.076320 4209 raft_consensus.cc:504] T 00000000000000000000000000000000 P a51b0e15e75942928e3fe4b922b50dd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a51b0e15e75942928e3fe4b922b50dd1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38799 } }
I20260929 10:37:24.076723 4209 leader_election.cc:302] T 00000000000000000000000000000000 P a51b0e15e75942928e3fe4b922b50dd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a51b0e15e75942928e3fe4b922b50dd1; no voters:
I20260929 10:37:24.077068 4209 leader_election.cc:288] T 00000000000000000000000000000000 P a51b0e15e75942928e3fe4b922b50dd1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:37:24.077160 4214 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a51b0e15e75942928e3fe4b922b50dd1 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:37:24.077392 4214 raft_consensus.cc:686] T 00000000000000000000000000000000 P a51b0e15e75942928e3fe4b922b50dd1 [term 1 LEADER]: Becoming Leader. State: Replica: a51b0e15e75942928e3fe4b922b50dd1, State: Running, Role: LEADER
I20260929 10:37:24.077783 4214 consensus_queue.cc:237] T 00000000000000000000000000000000 P a51b0e15e75942928e3fe4b922b50dd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a51b0e15e75942928e3fe4b922b50dd1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38799 } }
I20260929 10:37:24.078596 4209 sys_catalog.cc:564] T 00000000000000000000000000000000 P a51b0e15e75942928e3fe4b922b50dd1 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:37:24.079774 4216 sys_catalog.cc:455] T 00000000000000000000000000000000 P a51b0e15e75942928e3fe4b922b50dd1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a51b0e15e75942928e3fe4b922b50dd1. Latest consensus state: current_term: 1 leader_uuid: "a51b0e15e75942928e3fe4b922b50dd1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a51b0e15e75942928e3fe4b922b50dd1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38799 } } }
I20260929 10:37:24.079774 4215 sys_catalog.cc:455] T 00000000000000000000000000000000 P a51b0e15e75942928e3fe4b922b50dd1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a51b0e15e75942928e3fe4b922b50dd1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a51b0e15e75942928e3fe4b922b50dd1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38799 } } }
I20260929 10:37:24.079936 4215 sys_catalog.cc:458] T 00000000000000000000000000000000 P a51b0e15e75942928e3fe4b922b50dd1 [sys.catalog]: This master's current role is: LEADER
I20260929 10:37:24.079936 4216 sys_catalog.cc:458] T 00000000000000000000000000000000 P a51b0e15e75942928e3fe4b922b50dd1 [sys.catalog]: This master's current role is: LEADER
I20260929 10:37:24.080368 4223 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:37:24.081823 4212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:24.066764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.185:54045 (local address 127.0.0.1:38799)
0929 10:37:24.067227 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:24.067242 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:24.067281 (+ 39us) server_negotiation.cc:407] Connection header received
0929 10:37:24.068042 (+ 761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:24.068062 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:24.068472 (+ 410us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:24.068872 (+ 400us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:24.069723 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:24.070878 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:24.071778 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:24.072270 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:24.072452 (+ 182us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:24.075512 (+ 3060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:24.075518 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:24.075532 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:37:24.075559 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:24.077615 (+ 2056us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:24.078225 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:24.078231 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:24.078237 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:24.078387 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:24.078837 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:24.078841 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:24.078843 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:24.079371 (+ 528us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:24.079635 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:24.080466 (+ 831us) server_negotiation.cc:299] Negotiation successful
0929 10:37:24.080731 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":140,"threads_started":1}
I20260929 10:37:24.083395 4223 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:37:24.084873 15689 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 10:37:24.092002 4223 catalog_manager.cc:1347] Generated new cluster ID: 00a972f207b647a1b99400e9d7f38f6e
I20260929 10:37:24.092068 4223 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:37:24.098852 4212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:24.087603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.185:57499 (local address 127.0.0.1:38799)
0929 10:37:24.087706 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:24.087712 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:24.088064 (+ 352us) server_negotiation.cc:407] Connection header received
0929 10:37:24.088182 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:24.088184 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:24.088225 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:24.088291 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:24.089134 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:24.089655 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:24.090879 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:24.091239 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:24.091347 (+ 108us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:24.095457 (+ 4110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:24.095460 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:24.095462 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:37:24.095484 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:24.097009 (+ 1525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:24.097506 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:24.097517 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:24.097519 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:24.097592 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:24.097935 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:24.097943 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:24.097945 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:24.098110 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:24.098283 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:24.098597 (+ 314us) server_negotiation.cc:299] Negotiation successful
0929 10:37:24.098694 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260929 10:37:24.106365 4223 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:37:24.107493 4223 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:37:24.113999 4223 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a51b0e15e75942928e3fe4b922b50dd1: Generated new TSK 0
I20260929 10:37:24.114837 4223 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 10:37:24.123994 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 4141
W20260929 10:37:24.129861 4126 connection.cc:535] client connection to 127.0.0.1:38799 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38799): Connection reset by peer (error 104)
2026-09-29T10:37:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1418 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790677981592896-15689-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:37:24 dist-test-slave-vg89 krb5kdc[4240](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:37:24 dist-test-slave-vg89 krb5kdc[4240](info): set up 2 sockets
Sep 29 10:37:24 dist-test-slave-vg89 krb5kdc[4240](info): commencing operation
krb5kdc: starting...
W20260929 10:37:25.159037 15689 mini_kdc.cc:121] Time spent starting KDC: real 1.014s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Sep 29 10:37:25 dist-test-slave-vg89 krb5kdc[4240](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678245, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:37:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:37:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790677981592896-15689-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790677981592896-15689-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 10:37:25.286082 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790677981592896-15689-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790677981592896-15689-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790677981592896-15689-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36993
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790677981592896-15689-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36993
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:37:25.396539 4253 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:37:25.396838 4253 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:37:25.396881 4253 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:37:25.400957 4253 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:37:25.401015 4253 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:37:25.401034 4253 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:37:25.401051 4253 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:37:25.401101 4253 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:37:25.406386 4253 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790677981592896-15689-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790677981592896-15689-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36993
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790677981592896-15689-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36993
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4253
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790677981592896-15689-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:37:25.407595 4253 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:37:25.408696 4253 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:37:25.417413 4261 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:25.417383 4259 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:25.417448 4258 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:37:25.417624 4253 server_base.cc:1034] running on GCE node
I20260929 10:37:25.418286 4253 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:37:25.419492 4253 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:37:25.420702 4253 hybrid_clock.cc:648] HybridClock initialized: now 1790678245420678 us; error 31 us; skew 500 ppm
Sep 29 10:37:25 dist-test-slave-vg89 krb5kdc[4240](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678245, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:37:25.427829 4253 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 10:37:25.428879 4253 webserver.cc:460] Webserver started at http://127.0.0.1:41259/ using document root <none> and password file <none>
I20260929 10:37:25.429514 4253 fs_manager.cc:359] Metadata directory not provided
I20260929 10:37:25.429637 4253 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:37:25.429879 4253 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:37:25.432215 4253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790677981592896-15689-0/minicluster-data/master-0/data/instance:
uuid: "9a81c7b697c14af98ad6efa727c21d60"
format_stamp: "Formatted at 2026-09-29 10:37:25 on dist-test-slave-vg89"
server_key: "7accf3eb548f0b7892301e9f26fd75bd"
server_key_iv: "cca6d3622ac77765557b956e30765ae7"
server_key_version: "encryptionkey@0"
I20260929 10:37:25.432844 4253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790677981592896-15689-0/minicluster-data/master-0/wal/instance:
uuid: "9a81c7b697c14af98ad6efa727c21d60"
format_stamp: "Formatted at 2026-09-29 10:37:25 on dist-test-slave-vg89"
server_key: "7accf3eb548f0b7892301e9f26fd75bd"
server_key_iv: "cca6d3622ac77765557b956e30765ae7"
server_key_version: "encryptionkey@0"
I20260929 10:37:25.437000 4253 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 10:37:25.439814 4268 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:25.441180 4253 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 10:37:25.441339 4253 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790677981592896-15689-0/minicluster-data/master-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790677981592896-15689-0/minicluster-data/master-0/wal
uuid: "9a81c7b697c14af98ad6efa727c21d60"
format_stamp: "Formatted at 2026-09-29 10:37:25 on dist-test-slave-vg89"
server_key: "7accf3eb548f0b7892301e9f26fd75bd"
server_key_iv: "cca6d3622ac77765557b956e30765ae7"
server_key_version: "encryptionkey@0"
I20260929 10:37:25.441466 4253 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790677981592896-15689-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790677981592896-15689-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790677981592896-15689-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:37:25.463333 4253 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:37:25.466563 4253 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:37:25.466768 4253 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:37:25.474013 4253 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36993
I20260929 10:37:25.474083 4320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36993 every 8 connection(s)
I20260929 10:37:25.475085 4253 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
I20260929 10:37:25.478253 4321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:37:25.482973 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 4253
I20260929 10:37:25.483150 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790677981592896-15689-0/minicluster-data/master-0/wal/instance
I20260929 10:37:25.483500 15689 external_mini_cluster.cc:1428] Setting key 50e6d9c17ea52152b81a34b50cd75f97
I20260929 10:37:25.485291 4321 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a81c7b697c14af98ad6efa727c21d60: Bootstrap starting.
I20260929 10:37:25.487608 4321 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9a81c7b697c14af98ad6efa727c21d60: Neither blocks nor log segments found. Creating new log.
I20260929 10:37:25.488375 4321 log.cc:824] T 00000000000000000000000000000000 P 9a81c7b697c14af98ad6efa727c21d60: Log is configured to *not* fsync() on all Append() calls
I20260929 10:37:25.490643 4321 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a81c7b697c14af98ad6efa727c21d60: No bootstrap required, opened a new log
I20260929 10:37:25.493211 4324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:25.484966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.185:36043 (local address 127.0.0.1:36993)
0929 10:37:25.485473 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:25.485489 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:25.485526 (+ 37us) server_negotiation.cc:407] Connection header received
0929 10:37:25.486176 (+ 650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:25.486193 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:25.486602 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:25.486947 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:25.487359 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:25.488400 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:25.489662 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:25.490128 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:25.490251 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:25.491304 (+ 1053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:25.491309 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:25.491321 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:37:25.491699 (+ 378us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:25.491751 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:37:25.491773 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:25.491892 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:25.492106 (+ 214us) server_negotiation.cc:299] Negotiation successful
0929 10:37:25.492266 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":145,"threads_started":1}
I20260929 10:37:25.494124 4321 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9a81c7b697c14af98ad6efa727c21d60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a81c7b697c14af98ad6efa727c21d60" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36993 } }
I20260929 10:37:25.494344 4321 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9a81c7b697c14af98ad6efa727c21d60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:37:25.494438 4321 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9a81c7b697c14af98ad6efa727c21d60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a81c7b697c14af98ad6efa727c21d60, State: Initialized, Role: FOLLOWER
I20260929 10:37:25.494935 4321 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9a81c7b697c14af98ad6efa727c21d60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a81c7b697c14af98ad6efa727c21d60" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36993 } }
I20260929 10:37:25.495070 4321 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9a81c7b697c14af98ad6efa727c21d60 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:37:25.495163 4321 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9a81c7b697c14af98ad6efa727c21d60 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:37:25.495273 4321 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9a81c7b697c14af98ad6efa727c21d60 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:37:25.496269 15689 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 10:37:25.496376 4321 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9a81c7b697c14af98ad6efa727c21d60 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a81c7b697c14af98ad6efa727c21d60" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36993 } }
I20260929 10:37:25.496735 4321 leader_election.cc:302] T 00000000000000000000000000000000 P 9a81c7b697c14af98ad6efa727c21d60 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9a81c7b697c14af98ad6efa727c21d60; no voters:
I20260929 10:37:25.497078 4321 leader_election.cc:288] T 00000000000000000000000000000000 P 9a81c7b697c14af98ad6efa727c21d60 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:37:25.497154 4326 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9a81c7b697c14af98ad6efa727c21d60 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:37:25.497346 4326 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9a81c7b697c14af98ad6efa727c21d60 [term 1 LEADER]: Becoming Leader. State: Replica: 9a81c7b697c14af98ad6efa727c21d60, State: Running, Role: LEADER
I20260929 10:37:25.497660 4326 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9a81c7b697c14af98ad6efa727c21d60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a81c7b697c14af98ad6efa727c21d60" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36993 } }
I20260929 10:37:25.498484 4321 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9a81c7b697c14af98ad6efa727c21d60 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:37:25.498961 4328 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a81c7b697c14af98ad6efa727c21d60 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9a81c7b697c14af98ad6efa727c21d60. Latest consensus state: current_term: 1 leader_uuid: "9a81c7b697c14af98ad6efa727c21d60" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a81c7b697c14af98ad6efa727c21d60" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36993 } } }
I20260929 10:37:25.499092 4328 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a81c7b697c14af98ad6efa727c21d60 [sys.catalog]: This master's current role is: LEADER
I20260929 10:37:25.499342 4327 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a81c7b697c14af98ad6efa727c21d60 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9a81c7b697c14af98ad6efa727c21d60" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a81c7b697c14af98ad6efa727c21d60" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36993 } } }
I20260929 10:37:25.499450 4327 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a81c7b697c14af98ad6efa727c21d60 [sys.catalog]: This master's current role is: LEADER
I20260929 10:37:25.499573 4335 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:37:25.502939 4335 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:37:25.504477 4324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:25.499161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.185:39287 (local address 127.0.0.1:36993)
0929 10:37:25.499289 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:25.499294 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:25.499810 (+ 516us) server_negotiation.cc:407] Connection header received
0929 10:37:25.500016 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:25.500023 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:25.500073 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:25.500123 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:25.500573 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:25.501301 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:25.502511 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:25.502746 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:25.502866 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:25.504006 (+ 1140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:25.504009 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:25.504012 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:37:25.504024 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:25.504046 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:37:25.504056 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:25.504155 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:25.504277 (+ 122us) server_negotiation.cc:299] Negotiation successful
0929 10:37:25.504312 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260929 10:37:25.508302 4335 catalog_manager.cc:1347] Generated new cluster ID: 65340aa8585546e6ae28e94e8179e1d9
I20260929 10:37:25.508380 4335 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:37:25.516487 4335 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:37:25.517583 4335 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:37:25.523818 4335 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9a81c7b697c14af98ad6efa727c21d60: Generated new TSK 0
I20260929 10:37:25.524508 4335 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 10:37:25.530885 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 4253
W20260929 10:37:25.536728 4238 connection.cc:535] client connection to 127.0.0.1:36993 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36993): Connection reset by peer (error 104)
2026-09-29T10:37:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1406 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790677981592896-15689-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:37:25 dist-test-slave-vg89 krb5kdc[4352](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:37:25 dist-test-slave-vg89 krb5kdc[4352](info): set up 2 sockets
Sep 29 10:37:25 dist-test-slave-vg89 krb5kdc[4352](info): commencing operation
krb5kdc: starting...
W20260929 10:37:26.579676 15689 mini_kdc.cc:121] Time spent starting KDC: real 1.028s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Sep 29 10:37:26 dist-test-slave-vg89 krb5kdc[4352](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678246, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:37:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:37:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790677981592896-15689-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790677981592896-15689-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 10:37:26.705981 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790677981592896-15689-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790677981592896-15689-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790677981592896-15689-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35061
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790677981592896-15689-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35061
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:37:26.817654 4365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:37:26.817988 4365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:37:26.818066 4365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:37:26.822121 4365 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:37:26.822232 4365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:37:26.822268 4365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:37:26.822305 4365 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:37:26.822335 4365 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:37:26.827667 4365 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790677981592896-15689-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790677981592896-15689-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35061
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790677981592896-15689-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35061
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4365
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790677981592896-15689-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:37:26.828817 4365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:37:26.829924 4365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:37:26.838831 4371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:26.838828 4370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:37:26.838928 4365 server_base.cc:1034] running on GCE node
W20260929 10:37:26.838828 4373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:37:26.839633 4365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:37:26.840858 4365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:37:26.842026 4365 hybrid_clock.cc:648] HybridClock initialized: now 1790678246842015 us; error 29 us; skew 500 ppm
Sep 29 10:37:26 dist-test-slave-vg89 krb5kdc[4352](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678246, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:37:26.844929 4365 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 10:37:26.845916 4365 webserver.cc:460] Webserver started at http://127.0.0.1:43873/ using document root <none> and password file <none>
I20260929 10:37:26.846572 4365 fs_manager.cc:359] Metadata directory not provided
I20260929 10:37:26.846668 4365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:37:26.846915 4365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:37:26.849030 4365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790677981592896-15689-0/minicluster-data/master-0/data/instance:
uuid: "5d167e13f24249598bf3bf9a9669127d"
format_stamp: "Formatted at 2026-09-29 10:37:26 on dist-test-slave-vg89"
server_key: "900c00c4934aa94b4831539e32346972"
server_key_iv: "104cd174b1fb8b078171abacd11fee82"
server_key_version: "encryptionkey@0"
I20260929 10:37:26.849624 4365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790677981592896-15689-0/minicluster-data/master-0/wal/instance:
uuid: "5d167e13f24249598bf3bf9a9669127d"
format_stamp: "Formatted at 2026-09-29 10:37:26 on dist-test-slave-vg89"
server_key: "900c00c4934aa94b4831539e32346972"
server_key_iv: "104cd174b1fb8b078171abacd11fee82"
server_key_version: "encryptionkey@0"
I20260929 10:37:26.853574 4365 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260929 10:37:26.856161 4380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:26.857388 4365 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260929 10:37:26.857532 4365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790677981592896-15689-0/minicluster-data/master-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790677981592896-15689-0/minicluster-data/master-0/wal
uuid: "5d167e13f24249598bf3bf9a9669127d"
format_stamp: "Formatted at 2026-09-29 10:37:26 on dist-test-slave-vg89"
server_key: "900c00c4934aa94b4831539e32346972"
server_key_iv: "104cd174b1fb8b078171abacd11fee82"
server_key_version: "encryptionkey@0"
I20260929 10:37:26.857647 4365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790677981592896-15689-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790677981592896-15689-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790677981592896-15689-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:37:26.862306 4365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:37:26.865614 4365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:37:26.865803 4365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:37:26.872967 4365 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35061
I20260929 10:37:26.872985 4432 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35061 every 8 connection(s)
I20260929 10:37:26.873929 4365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
I20260929 10:37:26.876603 4433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:37:26.881901 4433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d167e13f24249598bf3bf9a9669127d: Bootstrap starting.
I20260929 10:37:26.882967 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 4365
I20260929 10:37:26.883113 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790677981592896-15689-0/minicluster-data/master-0/wal/instance
I20260929 10:37:26.883404 15689 external_mini_cluster.cc:1428] Setting key ba262aeeb9608361621b79b4181e4358
I20260929 10:37:26.884711 4433 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5d167e13f24249598bf3bf9a9669127d: Neither blocks nor log segments found. Creating new log.
I20260929 10:37:26.885454 4433 log.cc:824] T 00000000000000000000000000000000 P 5d167e13f24249598bf3bf9a9669127d: Log is configured to *not* fsync() on all Append() calls
I20260929 10:37:26.887689 4433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d167e13f24249598bf3bf9a9669127d: No bootstrap required, opened a new log
I20260929 10:37:26.888535 4436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:26.884510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.185:37061 (local address 127.0.0.1:35061)
0929 10:37:26.885003 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:26.885019 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:26.885054 (+ 35us) server_negotiation.cc:407] Connection header received
0929 10:37:26.885723 (+ 669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:26.885741 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:26.886124 (+ 383us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:26.886498 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:26.887303 (+ 805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:26.887307 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:26.887319 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:37:26.887329 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:26.887379 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:37:26.887397 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:26.887496 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:26.887642 (+ 146us) server_negotiation.cc:299] Negotiation successful
0929 10:37:26.887820 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":171,"threads_started":1}
I20260929 10:37:26.891355 15689 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 10:37:26.891754 4433 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d167e13f24249598bf3bf9a9669127d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d167e13f24249598bf3bf9a9669127d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35061 } }
I20260929 10:37:26.891984 4433 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5d167e13f24249598bf3bf9a9669127d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:37:26.892058 4433 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d167e13f24249598bf3bf9a9669127d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d167e13f24249598bf3bf9a9669127d, State: Initialized, Role: FOLLOWER
I20260929 10:37:26.892473 4433 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d167e13f24249598bf3bf9a9669127d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d167e13f24249598bf3bf9a9669127d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35061 } }
I20260929 10:37:26.892635 4433 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5d167e13f24249598bf3bf9a9669127d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:37:26.892709 4433 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5d167e13f24249598bf3bf9a9669127d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:37:26.892812 4433 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5d167e13f24249598bf3bf9a9669127d [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:37:26.894014 4433 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5d167e13f24249598bf3bf9a9669127d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d167e13f24249598bf3bf9a9669127d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35061 } }
I20260929 10:37:26.894441 4433 leader_election.cc:302] T 00000000000000000000000000000000 P 5d167e13f24249598bf3bf9a9669127d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5d167e13f24249598bf3bf9a9669127d; no voters:
I20260929 10:37:26.894810 4433 leader_election.cc:288] T 00000000000000000000000000000000 P 5d167e13f24249598bf3bf9a9669127d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:37:26.894917 4443 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5d167e13f24249598bf3bf9a9669127d [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:37:26.895128 4443 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5d167e13f24249598bf3bf9a9669127d [term 1 LEADER]: Becoming Leader. State: Replica: 5d167e13f24249598bf3bf9a9669127d, State: Running, Role: LEADER
I20260929 10:37:26.895460 4436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:26.893501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.185:50111 (local address 127.0.0.1:35061)
0929 10:37:26.893605 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:26.893615 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:26.893829 (+ 214us) server_negotiation.cc:407] Connection header received
0929 10:37:26.893958 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:26.893960 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:26.893987 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:26.894050 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:26.894811 (+ 761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:26.894815 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:26.894818 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:37:26.894831 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:26.894859 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:37:26.894873 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:26.894963 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:26.895302 (+ 339us) server_negotiation.cc:299] Negotiation successful
0929 10:37:26.895341 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260929 10:37:26.895449 4443 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5d167e13f24249598bf3bf9a9669127d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d167e13f24249598bf3bf9a9669127d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35061 } }
I20260929 10:37:26.896106 4433 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d167e13f24249598bf3bf9a9669127d [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:37:26.897387 4444 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d167e13f24249598bf3bf9a9669127d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5d167e13f24249598bf3bf9a9669127d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d167e13f24249598bf3bf9a9669127d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35061 } } }
I20260929 10:37:26.897522 4444 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d167e13f24249598bf3bf9a9669127d [sys.catalog]: This master's current role is: LEADER
I20260929 10:37:26.897857 4445 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d167e13f24249598bf3bf9a9669127d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5d167e13f24249598bf3bf9a9669127d. Latest consensus state: current_term: 1 leader_uuid: "5d167e13f24249598bf3bf9a9669127d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d167e13f24249598bf3bf9a9669127d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35061 } } }
I20260929 10:37:26.897961 4445 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d167e13f24249598bf3bf9a9669127d [sys.catalog]: This master's current role is: LEADER
W20260929 10:37:26.900940 4458 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5d167e13f24249598bf3bf9a9669127d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 10:37:26.901015 4458 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 10:37:26.901141 4453 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:37:26.902130 4453 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:37:26.907575 4453 catalog_manager.cc:1347] Generated new cluster ID: e6ea8410a9ad4a93ad870d7eb9088d54
I20260929 10:37:26.907639 4453 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:37:26.946472 4453 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:37:26.946877 4453 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:37:26.953238 4453 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5d167e13f24249598bf3bf9a9669127d: Generated new TSK 0
I20260929 10:37:26.953976 4453 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 10:37:26.969887 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 4365
2026-09-29T10:37:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1438 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790677981592896-15689-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:37:27 dist-test-slave-vg89 krb5kdc[4464](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:37:27 dist-test-slave-vg89 krb5kdc[4464](info): set up 2 sockets
Sep 29 10:37:27 dist-test-slave-vg89 krb5kdc[4464](info): commencing operation
krb5kdc: starting...
W20260929 10:37:28.015045 15689 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Sep 29 10:37:28 dist-test-slave-vg89 krb5kdc[4464](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678248, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:37:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:37:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790677981592896-15689-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790677981592896-15689-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 10:37:28.139817 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790677981592896-15689-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790677981592896-15689-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790677981592896-15689-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39585
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790677981592896-15689-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39585
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:37:28.252516 4477 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:37:28.252805 4477 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:37:28.252902 4477 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:37:28.256878 4477 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:37:28.256965 4477 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:37:28.257004 4477 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:37:28.257040 4477 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:37:28.257073 4477 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:37:28.262262 4477 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790677981592896-15689-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790677981592896-15689-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39585
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790677981592896-15689-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39585
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4477
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790677981592896-15689-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:37:28.263321 4477 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:37:28.264344 4477 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:37:28.273507 4483 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:28.273525 4485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:28.273527 4482 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:37:28.273784 4477 server_base.cc:1034] running on GCE node
I20260929 10:37:28.274363 4477 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:37:28.275444 4477 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:37:28.276628 4477 hybrid_clock.cc:648] HybridClock initialized: now 1790678248276607 us; error 31 us; skew 500 ppm
Sep 29 10:37:28 dist-test-slave-vg89 krb5kdc[4464](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678248, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:37:28.279361 4477 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 10:37:28.280262 4477 webserver.cc:460] Webserver started at http://127.0.0.1:46707/ using document root <none> and password file <none>
I20260929 10:37:28.280799 4477 fs_manager.cc:359] Metadata directory not provided
I20260929 10:37:28.280897 4477 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:37:28.281119 4477 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:37:28.283169 4477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790677981592896-15689-0/minicluster-data/master-0/data/instance:
uuid: "125e0ee0d4834ffdae007c06570e8ea4"
format_stamp: "Formatted at 2026-09-29 10:37:28 on dist-test-slave-vg89"
server_key: "b69144012eec51f05b1fbf178e10968b"
server_key_iv: "c6cafe3315f6c273df0c471c72d0de5d"
server_key_version: "encryptionkey@0"
I20260929 10:37:28.283723 4477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790677981592896-15689-0/minicluster-data/master-0/wal/instance:
uuid: "125e0ee0d4834ffdae007c06570e8ea4"
format_stamp: "Formatted at 2026-09-29 10:37:28 on dist-test-slave-vg89"
server_key: "b69144012eec51f05b1fbf178e10968b"
server_key_iv: "c6cafe3315f6c273df0c471c72d0de5d"
server_key_version: "encryptionkey@0"
I20260929 10:37:28.287381 4477 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260929 10:37:28.289733 4492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:28.290985 4477 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 10:37:28.291118 4477 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790677981592896-15689-0/minicluster-data/master-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790677981592896-15689-0/minicluster-data/master-0/wal
uuid: "125e0ee0d4834ffdae007c06570e8ea4"
format_stamp: "Formatted at 2026-09-29 10:37:28 on dist-test-slave-vg89"
server_key: "b69144012eec51f05b1fbf178e10968b"
server_key_iv: "c6cafe3315f6c273df0c471c72d0de5d"
server_key_version: "encryptionkey@0"
I20260929 10:37:28.291229 4477 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790677981592896-15689-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790677981592896-15689-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790677981592896-15689-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:37:28.307255 4477 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:37:28.310050 4477 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:37:28.310253 4477 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:37:28.317929 4477 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39585
I20260929 10:37:28.317982 4544 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39585 every 8 connection(s)
I20260929 10:37:28.318940 4477 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
I20260929 10:37:28.321666 4545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:37:28.326480 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 4477
I20260929 10:37:28.326648 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790677981592896-15689-0/minicluster-data/master-0/wal/instance
I20260929 10:37:28.326972 15689 external_mini_cluster.cc:1428] Setting key 9cbb6e2b04c67bda7135953da43abca1
I20260929 10:37:28.327296 4545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 125e0ee0d4834ffdae007c06570e8ea4: Bootstrap starting.
I20260929 10:37:28.329741 4545 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 125e0ee0d4834ffdae007c06570e8ea4: Neither blocks nor log segments found. Creating new log.
I20260929 10:37:28.330519 4545 log.cc:824] T 00000000000000000000000000000000 P 125e0ee0d4834ffdae007c06570e8ea4: Log is configured to *not* fsync() on all Append() calls
I20260929 10:37:28.332352 4545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 125e0ee0d4834ffdae007c06570e8ea4: No bootstrap required, opened a new log
Sep 29 10:37:28 dist-test-slave-vg89 krb5kdc[4464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678248, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260929 10:37:28.336359 4545 raft_consensus.cc:348] T 00000000000000000000000000000000 P 125e0ee0d4834ffdae007c06570e8ea4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "125e0ee0d4834ffdae007c06570e8ea4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39585 } }
I20260929 10:37:28.336571 4545 raft_consensus.cc:374] T 00000000000000000000000000000000 P 125e0ee0d4834ffdae007c06570e8ea4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:37:28.336637 4545 raft_consensus.cc:729] T 00000000000000000000000000000000 P 125e0ee0d4834ffdae007c06570e8ea4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 125e0ee0d4834ffdae007c06570e8ea4, State: Initialized, Role: FOLLOWER
I20260929 10:37:28.337128 4545 consensus_queue.cc:260] T 00000000000000000000000000000000 P 125e0ee0d4834ffdae007c06570e8ea4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "125e0ee0d4834ffdae007c06570e8ea4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39585 } }
I20260929 10:37:28.337281 4545 raft_consensus.cc:388] T 00000000000000000000000000000000 P 125e0ee0d4834ffdae007c06570e8ea4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:37:28.337414 4545 raft_consensus.cc:482] T 00000000000000000000000000000000 P 125e0ee0d4834ffdae007c06570e8ea4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:37:28.337543 4545 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 125e0ee0d4834ffdae007c06570e8ea4 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:37:28.338900 4545 raft_consensus.cc:504] T 00000000000000000000000000000000 P 125e0ee0d4834ffdae007c06570e8ea4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "125e0ee0d4834ffdae007c06570e8ea4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39585 } }
I20260929 10:37:28.339282 4545 leader_election.cc:302] T 00000000000000000000000000000000 P 125e0ee0d4834ffdae007c06570e8ea4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 125e0ee0d4834ffdae007c06570e8ea4; no voters:
I20260929 10:37:28.339630 4545 leader_election.cc:288] T 00000000000000000000000000000000 P 125e0ee0d4834ffdae007c06570e8ea4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:37:28.339744 4550 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 125e0ee0d4834ffdae007c06570e8ea4 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:37:28.340019 4550 raft_consensus.cc:686] T 00000000000000000000000000000000 P 125e0ee0d4834ffdae007c06570e8ea4 [term 1 LEADER]: Becoming Leader. State: Replica: 125e0ee0d4834ffdae007c06570e8ea4, State: Running, Role: LEADER
I20260929 10:37:28.340364 4550 consensus_queue.cc:237] T 00000000000000000000000000000000 P 125e0ee0d4834ffdae007c06570e8ea4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "125e0ee0d4834ffdae007c06570e8ea4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39585 } }
I20260929 10:37:28.341014 4545 sys_catalog.cc:564] T 00000000000000000000000000000000 P 125e0ee0d4834ffdae007c06570e8ea4 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:37:28.342123 4551 sys_catalog.cc:455] T 00000000000000000000000000000000 P 125e0ee0d4834ffdae007c06570e8ea4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "125e0ee0d4834ffdae007c06570e8ea4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "125e0ee0d4834ffdae007c06570e8ea4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39585 } } }
I20260929 10:37:28.342274 4551 sys_catalog.cc:458] T 00000000000000000000000000000000 P 125e0ee0d4834ffdae007c06570e8ea4 [sys.catalog]: This master's current role is: LEADER
I20260929 10:37:28.343065 4548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:28.328343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.185:45331 (local address 127.0.0.1:39585)
0929 10:37:28.328800 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:28.328816 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:28.328852 (+ 36us) server_negotiation.cc:407] Connection header received
0929 10:37:28.329492 (+ 640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:28.329509 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:28.329913 (+ 404us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:28.330264 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:28.331194 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:28.332177 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:28.333130 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:28.333623 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:28.333846 (+ 223us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:28.336793 (+ 2947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:28.336798 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:28.336812 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:37:28.336839 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:28.339214 (+ 2375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:28.339872 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:28.339881 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:28.339890 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:28.340033 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:28.340452 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:28.340455 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:28.340457 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:28.340945 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:28.341167 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:28.341556 (+ 389us) server_negotiation.cc:299] Negotiation successful
0929 10:37:28.341795 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":125,"threads_started":1}
I20260929 10:37:28.343513 4561 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:37:28.346078 15689 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 10:37:28.342851 4552 sys_catalog.cc:455] T 00000000000000000000000000000000 P 125e0ee0d4834ffdae007c06570e8ea4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 125e0ee0d4834ffdae007c06570e8ea4. Latest consensus state: current_term: 1 leader_uuid: "125e0ee0d4834ffdae007c06570e8ea4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "125e0ee0d4834ffdae007c06570e8ea4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39585 } } }
I20260929 10:37:28.346549 4552 sys_catalog.cc:458] T 00000000000000000000000000000000 P 125e0ee0d4834ffdae007c06570e8ea4 [sys.catalog]: This master's current role is: LEADER
I20260929 10:37:28.347184 4561 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:37:28.353240 4561 catalog_manager.cc:1347] Generated new cluster ID: fc63ba9401f0405ab78d21042a699c5a
I20260929 10:37:28.353299 4561 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:37:28.360198 4548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:28.348270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.185:40361 (local address 127.0.0.1:39585)
0929 10:37:28.348384 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:28.348388 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:28.350688 (+ 2300us) server_negotiation.cc:407] Connection header received
0929 10:37:28.350843 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:28.350846 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:28.350920 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:28.350996 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:28.351944 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:28.353147 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:28.354377 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:28.354758 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:28.354835 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:28.356692 (+ 1857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:28.356696 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:28.356698 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:37:28.356724 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:28.358313 (+ 1589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:28.358901 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:28.358906 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:28.358910 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:28.359002 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:28.359331 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:28.359334 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:28.359336 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:28.359543 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:28.359688 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:28.359943 (+ 255us) server_negotiation.cc:299] Negotiation successful
0929 10:37:28.360039 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260929 10:37:28.368454 4561 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:37:28.369349 4561 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:37:28.381042 4561 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 125e0ee0d4834ffdae007c06570e8ea4: Generated new TSK 0
I20260929 10:37:28.381860 4561 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 10:37:28.394608 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 4477
W20260929 10:37:28.400664 4462 connection.cc:535] client connection to 127.0.0.1:39585 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39585): Connection reset by peer (error 104)
2026-09-29T10:37:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1425 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790677981592896-15689-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:37:28 dist-test-slave-vg89 krb5kdc[4576](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:37:28 dist-test-slave-vg89 krb5kdc[4576](info): set up 2 sockets
Sep 29 10:37:28 dist-test-slave-vg89 krb5kdc[4576](info): commencing operation
krb5kdc: starting...
W20260929 10:37:29.430960 15689 mini_kdc.cc:121] Time spent starting KDC: real 1.016s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Sep 29 10:37:29 dist-test-slave-vg89 krb5kdc[4576](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678249, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:37:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:37:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790677981592896-15689-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790677981592896-15689-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 10:37:29.557195 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790677981592896-15689-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790677981592896-15689-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790677981592896-15689-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43165
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790677981592896-15689-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43165
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:37:29.668407 4589 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:37:29.668658 4589 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:37:29.668699 4589 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:37:29.672688 4589 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:37:29.672747 4589 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:37:29.672766 4589 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:37:29.672781 4589 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:37:29.672796 4589 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:37:29.678014 4589 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790677981592896-15689-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790677981592896-15689-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43165
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790677981592896-15689-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43165
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4589
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790677981592896-15689-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:37:29.679091 4589 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:37:29.680084 4589 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:37:29.688331 4594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:37:29.688458 4589 server_base.cc:1034] running on GCE node
W20260929 10:37:29.688432 4595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:29.688320 4597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:37:29.689172 4589 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:37:29.690210 4589 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:37:29.691366 4589 hybrid_clock.cc:648] HybridClock initialized: now 1790678249691343 us; error 43 us; skew 500 ppm
Sep 29 10:37:29 dist-test-slave-vg89 krb5kdc[4576](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678249, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:37:29.694058 4589 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 10:37:29.694993 4589 webserver.cc:460] Webserver started at http://127.0.0.1:46301/ using document root <none> and password file <none>
I20260929 10:37:29.695536 4589 fs_manager.cc:359] Metadata directory not provided
I20260929 10:37:29.695595 4589 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:37:29.695802 4589 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:37:29.697800 4589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790677981592896-15689-0/minicluster-data/master-0/data/instance:
uuid: "96806a3c7fe3482ca49004abe86bfc77"
format_stamp: "Formatted at 2026-09-29 10:37:29 on dist-test-slave-vg89"
server_key: "9452de7bc99585f16a6d17f3fe58006b"
server_key_iv: "62f7a1d1af804467189c5d7e86df84ce"
server_key_version: "encryptionkey@0"
I20260929 10:37:29.698380 4589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790677981592896-15689-0/minicluster-data/master-0/wal/instance:
uuid: "96806a3c7fe3482ca49004abe86bfc77"
format_stamp: "Formatted at 2026-09-29 10:37:29 on dist-test-slave-vg89"
server_key: "9452de7bc99585f16a6d17f3fe58006b"
server_key_iv: "62f7a1d1af804467189c5d7e86df84ce"
server_key_version: "encryptionkey@0"
I20260929 10:37:29.702080 4589 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260929 10:37:29.704525 4604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:29.705801 4589 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 10:37:29.705950 4589 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790677981592896-15689-0/minicluster-data/master-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790677981592896-15689-0/minicluster-data/master-0/wal
uuid: "96806a3c7fe3482ca49004abe86bfc77"
format_stamp: "Formatted at 2026-09-29 10:37:29 on dist-test-slave-vg89"
server_key: "9452de7bc99585f16a6d17f3fe58006b"
server_key_iv: "62f7a1d1af804467189c5d7e86df84ce"
server_key_version: "encryptionkey@0"
I20260929 10:37:29.706064 4589 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790677981592896-15689-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790677981592896-15689-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790677981592896-15689-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:37:29.719129 4589 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:37:29.721942 4589 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:37:29.722146 4589 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:37:29.729758 4589 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43165
I20260929 10:37:29.729785 4656 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43165 every 8 connection(s)
I20260929 10:37:29.730720 4589 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
I20260929 10:37:29.733438 4657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:37:29.734772 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 4589
I20260929 10:37:29.734913 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790677981592896-15689-0/minicluster-data/master-0/wal/instance
I20260929 10:37:29.735183 15689 external_mini_cluster.cc:1428] Setting key be78f451e3bfafdb40473dd9d4722a41
I20260929 10:37:29.739544 4657 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96806a3c7fe3482ca49004abe86bfc77: Bootstrap starting.
I20260929 10:37:29.741807 4657 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 96806a3c7fe3482ca49004abe86bfc77: Neither blocks nor log segments found. Creating new log.
I20260929 10:37:29.742601 4657 log.cc:824] T 00000000000000000000000000000000 P 96806a3c7fe3482ca49004abe86bfc77: Log is configured to *not* fsync() on all Append() calls
I20260929 10:37:29.744431 4660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:29.736600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.185:55037 (local address 127.0.0.1:43165)
0929 10:37:29.737081 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:29.737096 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:29.737131 (+ 35us) server_negotiation.cc:407] Connection header received
0929 10:37:29.737746 (+ 615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:29.737763 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:29.738123 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:29.738512 (+ 389us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:29.738943 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:29.739964 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:29.740889 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:29.741501 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:29.741673 (+ 172us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:29.742689 (+ 1016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:29.742695 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:29.742716 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:37:29.743209 (+ 493us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:29.743276 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:37:29.743308 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:29.743440 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:29.743594 (+ 154us) server_negotiation.cc:299] Negotiation successful
0929 10:37:29.743728 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":156,"threads_started":1}
I20260929 10:37:29.744705 4657 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96806a3c7fe3482ca49004abe86bfc77: No bootstrap required, opened a new log
I20260929 10:37:29.747334 15689 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 10:37:29.748672 4657 raft_consensus.cc:348] T 00000000000000000000000000000000 P 96806a3c7fe3482ca49004abe86bfc77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96806a3c7fe3482ca49004abe86bfc77" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43165 } }
I20260929 10:37:29.748909 4657 raft_consensus.cc:374] T 00000000000000000000000000000000 P 96806a3c7fe3482ca49004abe86bfc77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:37:29.748996 4657 raft_consensus.cc:729] T 00000000000000000000000000000000 P 96806a3c7fe3482ca49004abe86bfc77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96806a3c7fe3482ca49004abe86bfc77, State: Initialized, Role: FOLLOWER
I20260929 10:37:29.749467 4657 consensus_queue.cc:260] T 00000000000000000000000000000000 P 96806a3c7fe3482ca49004abe86bfc77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96806a3c7fe3482ca49004abe86bfc77" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43165 } }
I20260929 10:37:29.749620 4657 raft_consensus.cc:388] T 00000000000000000000000000000000 P 96806a3c7fe3482ca49004abe86bfc77 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:37:29.749691 4657 raft_consensus.cc:482] T 00000000000000000000000000000000 P 96806a3c7fe3482ca49004abe86bfc77 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:37:29.749815 4657 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 96806a3c7fe3482ca49004abe86bfc77 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:37:29.750923 4657 raft_consensus.cc:504] T 00000000000000000000000000000000 P 96806a3c7fe3482ca49004abe86bfc77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96806a3c7fe3482ca49004abe86bfc77" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43165 } }
I20260929 10:37:29.751281 4657 leader_election.cc:302] T 00000000000000000000000000000000 P 96806a3c7fe3482ca49004abe86bfc77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 96806a3c7fe3482ca49004abe86bfc77; no voters:
I20260929 10:37:29.751590 4657 leader_election.cc:288] T 00000000000000000000000000000000 P 96806a3c7fe3482ca49004abe86bfc77 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:37:29.751705 4667 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 96806a3c7fe3482ca49004abe86bfc77 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:37:29.751960 4667 raft_consensus.cc:686] T 00000000000000000000000000000000 P 96806a3c7fe3482ca49004abe86bfc77 [term 1 LEADER]: Becoming Leader. State: Replica: 96806a3c7fe3482ca49004abe86bfc77, State: Running, Role: LEADER
I20260929 10:37:29.752292 4667 consensus_queue.cc:237] T 00000000000000000000000000000000 P 96806a3c7fe3482ca49004abe86bfc77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96806a3c7fe3482ca49004abe86bfc77" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43165 } }
I20260929 10:37:29.752857 4657 sys_catalog.cc:564] T 00000000000000000000000000000000 P 96806a3c7fe3482ca49004abe86bfc77 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:37:29.754217 4668 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96806a3c7fe3482ca49004abe86bfc77 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "96806a3c7fe3482ca49004abe86bfc77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96806a3c7fe3482ca49004abe86bfc77" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43165 } } }
I20260929 10:37:29.754356 4668 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96806a3c7fe3482ca49004abe86bfc77 [sys.catalog]: This master's current role is: LEADER
I20260929 10:37:29.754730 4669 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96806a3c7fe3482ca49004abe86bfc77 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 96806a3c7fe3482ca49004abe86bfc77. Latest consensus state: current_term: 1 leader_uuid: "96806a3c7fe3482ca49004abe86bfc77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96806a3c7fe3482ca49004abe86bfc77" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43165 } } }
I20260929 10:37:29.754896 4669 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96806a3c7fe3482ca49004abe86bfc77 [sys.catalog]: This master's current role is: LEADER
I20260929 10:37:29.755084 4660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:29.749293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.185:47805 (local address 127.0.0.1:43165)
0929 10:37:29.749400 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:29.749404 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:29.749615 (+ 211us) server_negotiation.cc:407] Connection header received
0929 10:37:29.749788 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:29.749791 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:29.749846 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:29.749940 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:29.750201 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:29.750978 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:29.752156 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:29.752489 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:29.752562 (+ 73us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:29.754376 (+ 1814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:29.754380 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:29.754383 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:37:29.754425 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:29.754455 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:37:29.754471 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:29.754541 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:29.754816 (+ 275us) server_negotiation.cc:299] Negotiation successful
0929 10:37:29.754857 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260929 10:37:29.756071 4676 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:37:29.758527 4676 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:37:29.763736 4676 catalog_manager.cc:1347] Generated new cluster ID: 92c2132997194643b1e720bc75703fbf
I20260929 10:37:29.763797 4676 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:37:29.773254 4676 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:37:29.774163 4676 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:37:29.785887 4676 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 96806a3c7fe3482ca49004abe86bfc77: Generated new TSK 0
I20260929 10:37:29.786624 4676 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 10:37:29.792970 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 4589
W20260929 10:37:29.798763 4574 connection.cc:535] client connection to 127.0.0.1:43165 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43165): Connection reset by peer (error 104)
2026-09-29T10:37:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1398 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790677981592896-15689-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:37:29 dist-test-slave-vg89 krb5kdc[4688](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:37:29 dist-test-slave-vg89 krb5kdc[4688](info): set up 2 sockets
Sep 29 10:37:29 dist-test-slave-vg89 krb5kdc[4688](info): commencing operation
krb5kdc: starting...
W20260929 10:37:30.831027 15689 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Sep 29 10:37:30 dist-test-slave-vg89 krb5kdc[4688](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678250, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:37:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:37:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790677981592896-15689-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790677981592896-15689-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 10:37:30.958638 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790677981592896-15689-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790677981592896-15689-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790677981592896-15689-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38259
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790677981592896-15689-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38259
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:37:31.070151 4701 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:37:31.070511 4701 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:37:31.070632 4701 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:37:31.074653 4701 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:37:31.074755 4701 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:37:31.074805 4701 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:37:31.074857 4701 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:37:31.074894 4701 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:37:31.080214 4701 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790677981592896-15689-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790677981592896-15689-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38259
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790677981592896-15689-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38259
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4701
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790677981592896-15689-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:37:31.081348 4701 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:37:31.082418 4701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:37:31.091151 4706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:31.091198 4709 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:31.091198 4707 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:37:31.091544 4701 server_base.cc:1034] running on GCE node
I20260929 10:37:31.092135 4701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:37:31.093271 4701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:37:31.094482 4701 hybrid_clock.cc:648] HybridClock initialized: now 1790678251094459 us; error 32 us; skew 500 ppm
Sep 29 10:37:31 dist-test-slave-vg89 krb5kdc[4688](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678251, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:37:31.101191 4701 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 10:37:31.102121 4701 webserver.cc:460] Webserver started at http://127.0.0.1:38711/ using document root <none> and password file <none>
I20260929 10:37:31.102756 4701 fs_manager.cc:359] Metadata directory not provided
I20260929 10:37:31.102880 4701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:37:31.103124 4701 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:37:31.105291 4701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790677981592896-15689-0/minicluster-data/master-0/data/instance:
uuid: "42ab7ccf188a4304b4818932a1c7b36e"
format_stamp: "Formatted at 2026-09-29 10:37:31 on dist-test-slave-vg89"
server_key: "458ddd51975e74feadf4e414aa863f81"
server_key_iv: "83d195fc882b05e68cb2ba17935aafaf"
server_key_version: "encryptionkey@0"
I20260929 10:37:31.105865 4701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790677981592896-15689-0/minicluster-data/master-0/wal/instance:
uuid: "42ab7ccf188a4304b4818932a1c7b36e"
format_stamp: "Formatted at 2026-09-29 10:37:31 on dist-test-slave-vg89"
server_key: "458ddd51975e74feadf4e414aa863f81"
server_key_iv: "83d195fc882b05e68cb2ba17935aafaf"
server_key_version: "encryptionkey@0"
I20260929 10:37:31.109624 4701 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260929 10:37:31.112188 4716 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:31.113466 4701 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260929 10:37:31.113616 4701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790677981592896-15689-0/minicluster-data/master-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790677981592896-15689-0/minicluster-data/master-0/wal
uuid: "42ab7ccf188a4304b4818932a1c7b36e"
format_stamp: "Formatted at 2026-09-29 10:37:31 on dist-test-slave-vg89"
server_key: "458ddd51975e74feadf4e414aa863f81"
server_key_iv: "83d195fc882b05e68cb2ba17935aafaf"
server_key_version: "encryptionkey@0"
I20260929 10:37:31.113740 4701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790677981592896-15689-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790677981592896-15689-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790677981592896-15689-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:37:31.118417 4701 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:37:31.121654 4701 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:37:31.121863 4701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:37:31.128926 4701 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38259
I20260929 10:37:31.128952 4768 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38259 every 8 connection(s)
I20260929 10:37:31.129933 4701 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
I20260929 10:37:31.132777 4769 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:37:31.135274 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 4701
I20260929 10:37:31.135416 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790677981592896-15689-0/minicluster-data/master-0/wal/instance
I20260929 10:37:31.135697 15689 external_mini_cluster.cc:1428] Setting key 6fa7f77bbd745ed487dece3e80ac15ab
I20260929 10:37:31.139065 4769 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42ab7ccf188a4304b4818932a1c7b36e: Bootstrap starting.
I20260929 10:37:31.141256 4772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:31.137122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.185:37271 (local address 127.0.0.1:38259)
0929 10:37:31.137629 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:31.137643 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:31.137691 (+ 48us) server_negotiation.cc:407] Connection header received
0929 10:37:31.138369 (+ 678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:31.138389 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:31.138792 (+ 403us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:31.139148 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:31.140021 (+ 873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:31.140026 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:31.140039 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:37:31.140049 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:31.140102 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:37:31.140122 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:31.140270 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:31.140355 (+ 85us) server_negotiation.cc:299] Negotiation successful
0929 10:37:31.140530 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":165,"threads_started":1}
I20260929 10:37:31.141664 4769 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 42ab7ccf188a4304b4818932a1c7b36e: Neither blocks nor log segments found. Creating new log.
I20260929 10:37:31.142498 4769 log.cc:824] T 00000000000000000000000000000000 P 42ab7ccf188a4304b4818932a1c7b36e: Log is configured to *not* fsync() on all Append() calls
I20260929 10:37:31.144287 15689 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 10:37:31.144768 4769 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42ab7ccf188a4304b4818932a1c7b36e: No bootstrap required, opened a new log
I20260929 10:37:31.148309 4772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:31.146512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.185:51377 (local address 127.0.0.1:38259)
0929 10:37:31.146645 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:31.146649 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:31.146832 (+ 183us) server_negotiation.cc:407] Connection header received
0929 10:37:31.146956 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:31.146958 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:31.146990 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:31.147055 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:31.147765 (+ 710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:31.147769 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:31.147772 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:37:31.147784 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:31.147811 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:37:31.147822 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:31.147919 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:31.148070 (+ 151us) server_negotiation.cc:299] Negotiation successful
0929 10:37:31.148110 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 10:37:31.149288 4769 raft_consensus.cc:348] T 00000000000000000000000000000000 P 42ab7ccf188a4304b4818932a1c7b36e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42ab7ccf188a4304b4818932a1c7b36e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38259 } }
I20260929 10:37:31.149550 4769 raft_consensus.cc:374] T 00000000000000000000000000000000 P 42ab7ccf188a4304b4818932a1c7b36e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:37:31.149608 4769 raft_consensus.cc:729] T 00000000000000000000000000000000 P 42ab7ccf188a4304b4818932a1c7b36e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42ab7ccf188a4304b4818932a1c7b36e, State: Initialized, Role: FOLLOWER
I20260929 10:37:31.150045 4769 consensus_queue.cc:260] T 00000000000000000000000000000000 P 42ab7ccf188a4304b4818932a1c7b36e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42ab7ccf188a4304b4818932a1c7b36e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38259 } }
I20260929 10:37:31.150198 4769 raft_consensus.cc:388] T 00000000000000000000000000000000 P 42ab7ccf188a4304b4818932a1c7b36e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:37:31.150285 4769 raft_consensus.cc:482] T 00000000000000000000000000000000 P 42ab7ccf188a4304b4818932a1c7b36e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:37:31.150386 4769 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 42ab7ccf188a4304b4818932a1c7b36e [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:37:31.151362 4769 raft_consensus.cc:504] T 00000000000000000000000000000000 P 42ab7ccf188a4304b4818932a1c7b36e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42ab7ccf188a4304b4818932a1c7b36e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38259 } }
I20260929 10:37:31.151711 4769 leader_election.cc:302] T 00000000000000000000000000000000 P 42ab7ccf188a4304b4818932a1c7b36e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42ab7ccf188a4304b4818932a1c7b36e; no voters:
I20260929 10:37:31.152029 4769 leader_election.cc:288] T 00000000000000000000000000000000 P 42ab7ccf188a4304b4818932a1c7b36e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:37:31.152189 4779 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 42ab7ccf188a4304b4818932a1c7b36e [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:37:31.152426 4779 raft_consensus.cc:686] T 00000000000000000000000000000000 P 42ab7ccf188a4304b4818932a1c7b36e [term 1 LEADER]: Becoming Leader. State: Replica: 42ab7ccf188a4304b4818932a1c7b36e, State: Running, Role: LEADER
I20260929 10:37:31.153134 4779 consensus_queue.cc:237] T 00000000000000000000000000000000 P 42ab7ccf188a4304b4818932a1c7b36e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42ab7ccf188a4304b4818932a1c7b36e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38259 } }
I20260929 10:37:31.153477 4769 sys_catalog.cc:564] T 00000000000000000000000000000000 P 42ab7ccf188a4304b4818932a1c7b36e [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:37:31.154951 4780 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42ab7ccf188a4304b4818932a1c7b36e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "42ab7ccf188a4304b4818932a1c7b36e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42ab7ccf188a4304b4818932a1c7b36e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38259 } } }
I20260929 10:37:31.154930 4781 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42ab7ccf188a4304b4818932a1c7b36e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 42ab7ccf188a4304b4818932a1c7b36e. Latest consensus state: current_term: 1 leader_uuid: "42ab7ccf188a4304b4818932a1c7b36e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42ab7ccf188a4304b4818932a1c7b36e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38259 } } }
I20260929 10:37:31.155130 4780 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42ab7ccf188a4304b4818932a1c7b36e [sys.catalog]: This master's current role is: LEADER
I20260929 10:37:31.155146 4781 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42ab7ccf188a4304b4818932a1c7b36e [sys.catalog]: This master's current role is: LEADER
I20260929 10:37:31.155529 4793 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:37:31.158612 4793 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:37:31.163574 4793 catalog_manager.cc:1347] Generated new cluster ID: 5637e32cef6d407bb5496162c85113a3
I20260929 10:37:31.163650 4793 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:37:31.172976 4793 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:37:31.173291 4793 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:37:31.188349 4793 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 42ab7ccf188a4304b4818932a1c7b36e: Generated new TSK 0
I20260929 10:37:31.189339 4793 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 10:37:31.202332 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 4701
2026-09-29T10:37:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1409 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790677981592896-15689-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:37:31 dist-test-slave-vg89 krb5kdc[4800](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:37:31 dist-test-slave-vg89 krb5kdc[4800](info): set up 2 sockets
Sep 29 10:37:31 dist-test-slave-vg89 krb5kdc[4800](info): commencing operation
krb5kdc: starting...
W20260929 10:37:32.238921 15689 mini_kdc.cc:121] Time spent starting KDC: real 1.016s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Sep 29 10:37:32 dist-test-slave-vg89 krb5kdc[4800](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678252, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:37:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:37:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.82.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab.
Entry for principal kudu/127.15.82.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab.
I20260929 10:37:32.367591 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790677981592896-15689-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790677981592896-15689-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790677981592896-15689-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:45099
--webserver_interface=127.15.82.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab
--principal=kudu/127.15.82.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:37683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.82.126:45099
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:37:32.481549 4813 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:37:32.481864 4813 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:37:32.481947 4813 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:37:32.485980 4813 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:37:32.486073 4813 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:37:32.486111 4813 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:37:32.486150 4813 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:37:32.486181 4813 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:37:32.491495 4813 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:37683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790677981592896-15689-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790677981592896-15689-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.82.126:45099
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab
--principal=kudu/127.15.82.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:45099
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.82.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4813
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790677981592896-15689-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:37:32.492574 4813 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:37:32.493717 4813 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:37:32.502116 4821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:32.502125 4818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:32.502163 4819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:37:32.502345 4813 server_base.cc:1034] running on GCE node
I20260929 10:37:32.503010 4813 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:37:32.504154 4813 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:37:32.505338 4813 hybrid_clock.cc:648] HybridClock initialized: now 1790678252505317 us; error 30 us; skew 500 ppm
Sep 29 10:37:32 dist-test-slave-vg89 krb5kdc[4800](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678252, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:37:32.508335 4813 init.cc:377] Logged in from keytab as kudu/127.15.82.126@KRBTEST.COM (short username kudu)
I20260929 10:37:32.509207 4813 webserver.cc:460] Webserver started at http://127.15.82.126:40541/ using document root <none> and password file <none>
I20260929 10:37:32.509800 4813 fs_manager.cc:359] Metadata directory not provided
I20260929 10:37:32.509865 4813 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:37:32.510102 4813 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:37:32.512195 4813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790677981592896-15689-0/minicluster-data/master-0/data/instance:
uuid: "b1d2d8a665914787b6c24fbc901eb6e7"
format_stamp: "Formatted at 2026-09-29 10:37:32 on dist-test-slave-vg89"
server_key: "228ae3cccc9635810002f7099da57d16"
server_key_iv: "fa9327fdc8847208c393c8eb05826941"
server_key_version: "encryptionkey@0"
I20260929 10:37:32.512774 4813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790677981592896-15689-0/minicluster-data/master-0/wal/instance:
uuid: "b1d2d8a665914787b6c24fbc901eb6e7"
format_stamp: "Formatted at 2026-09-29 10:37:32 on dist-test-slave-vg89"
server_key: "228ae3cccc9635810002f7099da57d16"
server_key_iv: "fa9327fdc8847208c393c8eb05826941"
server_key_version: "encryptionkey@0"
I20260929 10:37:32.516474 4813 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260929 10:37:32.518960 4828 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:32.520324 4813 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 10:37:32.520476 4813 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790677981592896-15689-0/minicluster-data/master-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790677981592896-15689-0/minicluster-data/master-0/wal
uuid: "b1d2d8a665914787b6c24fbc901eb6e7"
format_stamp: "Formatted at 2026-09-29 10:37:32 on dist-test-slave-vg89"
server_key: "228ae3cccc9635810002f7099da57d16"
server_key_iv: "fa9327fdc8847208c393c8eb05826941"
server_key_version: "encryptionkey@0"
I20260929 10:37:32.520593 4813 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790677981592896-15689-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790677981592896-15689-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790677981592896-15689-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:37:32.536662 4813 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:37:32.539703 4813 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:37:32.539923 4813 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:37:32.547554 4813 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.126:45099
I20260929 10:37:32.547613 4880 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.126:45099 every 8 connection(s)
I20260929 10:37:32.548592 4813 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
I20260929 10:37:32.551430 4881 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:37:32.554700 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 4813
I20260929 10:37:32.554868 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790677981592896-15689-0/minicluster-data/master-0/wal/instance
I20260929 10:37:32.555183 15689 external_mini_cluster.cc:1428] Setting key 08a0c9e6e6bc1fab2a28dd23b78f573c
I20260929 10:37:32.557274 4881 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1d2d8a665914787b6c24fbc901eb6e7: Bootstrap starting.
I20260929 10:37:32.559619 4881 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b1d2d8a665914787b6c24fbc901eb6e7: Neither blocks nor log segments found. Creating new log.
I20260929 10:37:32.560343 4881 log.cc:824] T 00000000000000000000000000000000 P b1d2d8a665914787b6c24fbc901eb6e7: Log is configured to *not* fsync() on all Append() calls
I20260929 10:37:32.562134 4881 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1d2d8a665914787b6c24fbc901eb6e7: No bootstrap required, opened a new log
Sep 29 10:37:32 dist-test-slave-vg89 krb5kdc[4800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678252, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:37:32.565860 4881 raft_consensus.cc:348] T 00000000000000000000000000000000 P b1d2d8a665914787b6c24fbc901eb6e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1d2d8a665914787b6c24fbc901eb6e7" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 45099 } }
I20260929 10:37:32.566128 4881 raft_consensus.cc:374] T 00000000000000000000000000000000 P b1d2d8a665914787b6c24fbc901eb6e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:37:32.566207 4881 raft_consensus.cc:729] T 00000000000000000000000000000000 P b1d2d8a665914787b6c24fbc901eb6e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1d2d8a665914787b6c24fbc901eb6e7, State: Initialized, Role: FOLLOWER
I20260929 10:37:32.566713 4881 consensus_queue.cc:260] T 00000000000000000000000000000000 P b1d2d8a665914787b6c24fbc901eb6e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1d2d8a665914787b6c24fbc901eb6e7" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 45099 } }
I20260929 10:37:32.566872 4881 raft_consensus.cc:388] T 00000000000000000000000000000000 P b1d2d8a665914787b6c24fbc901eb6e7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:37:32.566959 4881 raft_consensus.cc:482] T 00000000000000000000000000000000 P b1d2d8a665914787b6c24fbc901eb6e7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:37:32.567056 4881 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b1d2d8a665914787b6c24fbc901eb6e7 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:37:32.567905 4881 raft_consensus.cc:504] T 00000000000000000000000000000000 P b1d2d8a665914787b6c24fbc901eb6e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1d2d8a665914787b6c24fbc901eb6e7" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 45099 } }
I20260929 10:37:32.568241 4881 leader_election.cc:302] T 00000000000000000000000000000000 P b1d2d8a665914787b6c24fbc901eb6e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b1d2d8a665914787b6c24fbc901eb6e7; no voters:
I20260929 10:37:32.568568 4881 leader_election.cc:288] T 00000000000000000000000000000000 P b1d2d8a665914787b6c24fbc901eb6e7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:37:32.568673 4886 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b1d2d8a665914787b6c24fbc901eb6e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:37:32.568964 4886 raft_consensus.cc:686] T 00000000000000000000000000000000 P b1d2d8a665914787b6c24fbc901eb6e7 [term 1 LEADER]: Becoming Leader. State: Replica: b1d2d8a665914787b6c24fbc901eb6e7, State: Running, Role: LEADER
I20260929 10:37:32.569270 4886 consensus_queue.cc:237] T 00000000000000000000000000000000 P b1d2d8a665914787b6c24fbc901eb6e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1d2d8a665914787b6c24fbc901eb6e7" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 45099 } }
I20260929 10:37:32.569856 4881 sys_catalog.cc:564] T 00000000000000000000000000000000 P b1d2d8a665914787b6c24fbc901eb6e7 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:37:32.569883 4884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:32.556634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34729 (local address 127.15.82.126:45099)
0929 10:37:32.557044 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:32.557058 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:32.557092 (+ 34us) server_negotiation.cc:407] Connection header received
0929 10:37:32.557849 (+ 757us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:32.557867 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:32.558254 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:32.558621 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:32.559438 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:32.560519 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:32.561373 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:32.561861 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:32.562030 (+ 169us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:32.564652 (+ 2622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:32.564657 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:32.564670 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:37:32.564695 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:32.566611 (+ 1916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:32.567155 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:32.567161 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:32.567166 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:32.567283 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:32.567628 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:32.567631 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:32.567633 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:32.568018 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:32.568245 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:32.568676 (+ 431us) server_negotiation.cc:299] Negotiation successful
0929 10:37:32.568920 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":125,"threads_started":1}
I20260929 10:37:32.570895 4887 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1d2d8a665914787b6c24fbc901eb6e7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b1d2d8a665914787b6c24fbc901eb6e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1d2d8a665914787b6c24fbc901eb6e7" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 45099 } } }
I20260929 10:37:32.571101 4887 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1d2d8a665914787b6c24fbc901eb6e7 [sys.catalog]: This master's current role is: LEADER
I20260929 10:37:32.571382 4887 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1d2d8a665914787b6c24fbc901eb6e7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b1d2d8a665914787b6c24fbc901eb6e7. Latest consensus state: current_term: 1 leader_uuid: "b1d2d8a665914787b6c24fbc901eb6e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1d2d8a665914787b6c24fbc901eb6e7" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 45099 } } }
I20260929 10:37:32.571597 4887 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1d2d8a665914787b6c24fbc901eb6e7 [sys.catalog]: This master's current role is: LEADER
I20260929 10:37:32.572435 4895 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:37:32.573451 15689 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 10:37:32.575306 4895 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:37:32.581897 4895 catalog_manager.cc:1347] Generated new cluster ID: 69858980c41a4df2accd7a4a84e6ac78
I20260929 10:37:32.582654 4895 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:37:32.584388 4884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:32.575717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34679 (local address 127.15.82.126:45099)
0929 10:37:32.575837 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:32.575841 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:32.576091 (+ 250us) server_negotiation.cc:407] Connection header received
0929 10:37:32.576205 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:32.576209 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:32.576254 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:32.576346 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:32.577113 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:32.577914 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:32.579059 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:32.579452 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:32.579565 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:32.580971 (+ 1406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:32.580974 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:32.580977 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:37:32.581008 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:32.582480 (+ 1472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:32.583018 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:32.583021 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:32.583023 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:32.583117 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:32.583496 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:32.583499 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:32.583501 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:32.583719 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:32.583869 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:32.584127 (+ 258us) server_negotiation.cc:299] Negotiation successful
0929 10:37:32.584226 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260929 10:37:32.590022 4895 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:37:32.591168 4895 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:37:32.595453 4895 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b1d2d8a665914787b6c24fbc901eb6e7: Generated new TSK 0
I20260929 10:37:32.596199 4895 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 10:37:32.599958 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 4813
W20260929 10:37:32.605718 4798 connection.cc:535] client connection to 127.15.82.126:45099 recv error: Network error: failed to read from TLS socket (remote: 127.15.82.126:45099): Connection reset by peer (error 104)
2026-09-29T10:37:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1397 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790677981592896-15689-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:37:32 dist-test-slave-vg89 krb5kdc[4912](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:37:32 dist-test-slave-vg89 krb5kdc[4912](info): set up 2 sockets
Sep 29 10:37:32 dist-test-slave-vg89 krb5kdc[4912](info): commencing operation
krb5kdc: starting...
W20260929 10:37:33.635192 15689 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Sep 29 10:37:33 dist-test-slave-vg89 krb5kdc[4912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678253, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:37:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:37:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.82.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab.
Entry for principal kudu/127.15.82.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab.
I20260929 10:37:33.765971 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790677981592896-15689-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790677981592896-15689-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790677981592896-15689-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:36481
--webserver_interface=127.15.82.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab
--principal=kudu/127.15.82.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:35499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.82.126:36481
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:37:33.878304 4925 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:37:33.878690 4925 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:37:33.878782 4925 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:37:33.882793 4925 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:37:33.882885 4925 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:37:33.882925 4925 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:37:33.882979 4925 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:37:33.883018 4925 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:37:33.888742 4925 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:35499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790677981592896-15689-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790677981592896-15689-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.82.126:36481
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab
--principal=kudu/127.15.82.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:36481
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.82.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4925
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790677981592896-15689-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:37:33.889889 4925 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:37:33.890969 4925 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:37:33.899580 4933 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:33.899580 4930 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:33.899654 4931 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:37:33.899852 4925 server_base.cc:1034] running on GCE node
I20260929 10:37:33.900560 4925 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:37:33.901713 4925 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:37:33.902940 4925 hybrid_clock.cc:648] HybridClock initialized: now 1790678253902923 us; error 25 us; skew 500 ppm
Sep 29 10:37:33 dist-test-slave-vg89 krb5kdc[4912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678253, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:37:33.905817 4925 init.cc:377] Logged in from keytab as kudu/127.15.82.126@KRBTEST.COM (short username kudu)
I20260929 10:37:33.906772 4925 webserver.cc:460] Webserver started at http://127.15.82.126:45183/ using document root <none> and password file <none>
I20260929 10:37:33.907361 4925 fs_manager.cc:359] Metadata directory not provided
I20260929 10:37:33.907457 4925 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:37:33.907693 4925 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:37:33.909724 4925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790677981592896-15689-0/minicluster-data/master-0/data/instance:
uuid: "5582a4381c054b42981c35474e6fd4dc"
format_stamp: "Formatted at 2026-09-29 10:37:33 on dist-test-slave-vg89"
server_key: "8a2f83c0effddf1f63f128cf60ea50c0"
server_key_iv: "f9a19148ceb4e8b6f1f6a14976b36d81"
server_key_version: "encryptionkey@0"
I20260929 10:37:33.910298 4925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790677981592896-15689-0/minicluster-data/master-0/wal/instance:
uuid: "5582a4381c054b42981c35474e6fd4dc"
format_stamp: "Formatted at 2026-09-29 10:37:33 on dist-test-slave-vg89"
server_key: "8a2f83c0effddf1f63f128cf60ea50c0"
server_key_iv: "f9a19148ceb4e8b6f1f6a14976b36d81"
server_key_version: "encryptionkey@0"
I20260929 10:37:33.914157 4925 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260929 10:37:33.916687 4940 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:33.918001 4925 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 10:37:33.918151 4925 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790677981592896-15689-0/minicluster-data/master-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790677981592896-15689-0/minicluster-data/master-0/wal
uuid: "5582a4381c054b42981c35474e6fd4dc"
format_stamp: "Formatted at 2026-09-29 10:37:33 on dist-test-slave-vg89"
server_key: "8a2f83c0effddf1f63f128cf60ea50c0"
server_key_iv: "f9a19148ceb4e8b6f1f6a14976b36d81"
server_key_version: "encryptionkey@0"
I20260929 10:37:33.918265 4925 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790677981592896-15689-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790677981592896-15689-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790677981592896-15689-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:37:33.942579 4925 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:37:33.946062 4925 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:37:33.946276 4925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:37:33.953883 4925 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.126:36481
I20260929 10:37:33.953907 4992 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.126:36481 every 8 connection(s)
I20260929 10:37:33.954963 4925 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
I20260929 10:37:33.957764 4993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:37:33.963282 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 4925
I20260929 10:37:33.963410 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790677981592896-15689-0/minicluster-data/master-0/wal/instance
I20260929 10:37:33.963702 15689 external_mini_cluster.cc:1428] Setting key a005a9eac5d7f53549db02e54ac07aea
I20260929 10:37:33.963697 4993 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5582a4381c054b42981c35474e6fd4dc: Bootstrap starting.
I20260929 10:37:33.966315 4993 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5582a4381c054b42981c35474e6fd4dc: Neither blocks nor log segments found. Creating new log.
I20260929 10:37:33.967118 4993 log.cc:824] T 00000000000000000000000000000000 P 5582a4381c054b42981c35474e6fd4dc: Log is configured to *not* fsync() on all Append() calls
I20260929 10:37:33.969077 4993 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5582a4381c054b42981c35474e6fd4dc: No bootstrap required, opened a new log
I20260929 10:37:33.973335 4996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:33.965058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32833 (local address 127.15.82.126:36481)
0929 10:37:33.965513 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:33.965529 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:33.965564 (+ 35us) server_negotiation.cc:407] Connection header received
0929 10:37:33.966201 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:33.966219 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:33.966638 (+ 419us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:33.966984 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:33.967418 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:33.968366 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:33.969314 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:33.970025 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:33.970215 (+ 190us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:33.971094 (+ 879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:33.971101 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:33.971118 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:37:33.971662 (+ 544us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:33.971746 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:37:33.971778 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:33.971934 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:33.972145 (+ 211us) server_negotiation.cc:299] Negotiation successful
0929 10:37:33.972316 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":145,"threads_started":1}
I20260929 10:37:33.973816 4993 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5582a4381c054b42981c35474e6fd4dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5582a4381c054b42981c35474e6fd4dc" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 36481 } }
I20260929 10:37:33.974094 4993 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5582a4381c054b42981c35474e6fd4dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:37:33.974211 4993 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5582a4381c054b42981c35474e6fd4dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5582a4381c054b42981c35474e6fd4dc, State: Initialized, Role: FOLLOWER
I20260929 10:37:33.974714 4993 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5582a4381c054b42981c35474e6fd4dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5582a4381c054b42981c35474e6fd4dc" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 36481 } }
I20260929 10:37:33.974888 4993 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5582a4381c054b42981c35474e6fd4dc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:37:33.974979 4993 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5582a4381c054b42981c35474e6fd4dc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:37:33.975112 4993 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5582a4381c054b42981c35474e6fd4dc [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:37:33.976186 4993 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5582a4381c054b42981c35474e6fd4dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5582a4381c054b42981c35474e6fd4dc" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 36481 } }
I20260929 10:37:33.976426 15689 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 10:37:33.976575 4993 leader_election.cc:302] T 00000000000000000000000000000000 P 5582a4381c054b42981c35474e6fd4dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5582a4381c054b42981c35474e6fd4dc; no voters:
I20260929 10:37:33.976912 4993 leader_election.cc:288] T 00000000000000000000000000000000 P 5582a4381c054b42981c35474e6fd4dc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:37:33.976992 4998 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5582a4381c054b42981c35474e6fd4dc [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:37:33.977288 4998 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5582a4381c054b42981c35474e6fd4dc [term 1 LEADER]: Becoming Leader. State: Replica: 5582a4381c054b42981c35474e6fd4dc, State: Running, Role: LEADER
I20260929 10:37:33.977662 4998 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5582a4381c054b42981c35474e6fd4dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5582a4381c054b42981c35474e6fd4dc" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 36481 } }
I20260929 10:37:33.978361 4993 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5582a4381c054b42981c35474e6fd4dc [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:37:33.979029 4998 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5582a4381c054b42981c35474e6fd4dc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5582a4381c054b42981c35474e6fd4dc. Latest consensus state: current_term: 1 leader_uuid: "5582a4381c054b42981c35474e6fd4dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5582a4381c054b42981c35474e6fd4dc" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 36481 } } }
I20260929 10:37:33.979212 4998 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5582a4381c054b42981c35474e6fd4dc [sys.catalog]: This master's current role is: LEADER
I20260929 10:37:33.979527 4999 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5582a4381c054b42981c35474e6fd4dc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5582a4381c054b42981c35474e6fd4dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5582a4381c054b42981c35474e6fd4dc" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 36481 } } }
I20260929 10:37:33.979621 5009 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:37:33.979640 4999 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5582a4381c054b42981c35474e6fd4dc [sys.catalog]: This master's current role is: LEADER
I20260929 10:37:33.982857 5009 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:37:33.988729 4996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:33.983495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55279 (local address 127.15.82.126:36481)
0929 10:37:33.983631 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:33.983636 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:33.984087 (+ 451us) server_negotiation.cc:407] Connection header received
0929 10:37:33.984287 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:33.984291 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:33.984344 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:33.984456 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:33.984761 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:33.985522 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:33.986689 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:33.987052 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:33.987174 (+ 122us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:33.988167 (+ 993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:33.988172 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:33.988175 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:37:33.988190 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:33.988222 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:37:33.988239 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:33.988351 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:33.988526 (+ 175us) server_negotiation.cc:299] Negotiation successful
0929 10:37:33.988565 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260929 10:37:33.989363 5009 catalog_manager.cc:1347] Generated new cluster ID: 63dfa160164c4f42ace35e05aeda4854
I20260929 10:37:33.989460 5009 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:37:34.030884 5009 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:37:34.032023 5009 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:37:34.039417 5009 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5582a4381c054b42981c35474e6fd4dc: Generated new TSK 0
I20260929 10:37:34.040140 5009 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 10:37:34.046761 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 4925
W20260929 10:37:34.052893 4910 connection.cc:535] client connection to 127.15.82.126:36481 recv error: Network error: failed to read from TLS socket (remote: 127.15.82.126:36481): Connection reset by peer (error 104)
2026-09-29T10:37:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1447 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790677981592896-15689-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:37:34 dist-test-slave-vg89 krb5kdc[5024](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:37:34 dist-test-slave-vg89 krb5kdc[5024](info): set up 2 sockets
Sep 29 10:37:34 dist-test-slave-vg89 krb5kdc[5024](info): commencing operation
krb5kdc: starting...
W20260929 10:37:35.130980 15689 mini_kdc.cc:121] Time spent starting KDC: real 1.064s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Sep 29 10:37:35 dist-test-slave-vg89 krb5kdc[5024](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678255, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:37:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:37:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.82.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab.
Entry for principal kudu/127.15.82.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab.
I20260929 10:37:35.259426 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790677981592896-15689-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790677981592896-15689-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790677981592896-15689-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:34087
--webserver_interface=127.15.82.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab
--principal=kudu/127.15.82.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:43327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.82.126:34087
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:37:35.372426 5037 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:37:35.372691 5037 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:37:35.372735 5037 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:37:35.376878 5037 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:37:35.376937 5037 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:37:35.376957 5037 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:37:35.376973 5037 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:37:35.377010 5037 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:37:35.382261 5037 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:43327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790677981592896-15689-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790677981592896-15689-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.82.126:34087
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab
--principal=kudu/127.15.82.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:34087
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.82.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5037
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790677981592896-15689-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:37:35.383412 5037 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:37:35.384464 5037 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:37:35.392992 5042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:35.393034 5045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:37:35.393152 5037 server_base.cc:1034] running on GCE node
W20260929 10:37:35.393373 5043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:37:35.393934 5037 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:37:35.395128 5037 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:37:35.396345 5037 hybrid_clock.cc:648] HybridClock initialized: now 1790678255396325 us; error 36 us; skew 500 ppm
Sep 29 10:37:35 dist-test-slave-vg89 krb5kdc[5024](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678255, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:37:35.399307 5037 init.cc:377] Logged in from keytab as kudu/127.15.82.126@KRBTEST.COM (short username kudu)
I20260929 10:37:35.400218 5037 webserver.cc:460] Webserver started at http://127.15.82.126:36407/ using document root <none> and password file <none>
I20260929 10:37:35.400820 5037 fs_manager.cc:359] Metadata directory not provided
I20260929 10:37:35.400914 5037 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:37:35.401150 5037 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:37:35.403304 5037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790677981592896-15689-0/minicluster-data/master-0/data/instance:
uuid: "9ad08b1c51af43d0bf6ae39a73efaf60"
format_stamp: "Formatted at 2026-09-29 10:37:35 on dist-test-slave-vg89"
server_key: "b77ab451f21f7fdea337398dfcba1c8b"
server_key_iv: "9fe7d5fad3743c00b5b9627808eae08d"
server_key_version: "encryptionkey@0"
I20260929 10:37:35.403872 5037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790677981592896-15689-0/minicluster-data/master-0/wal/instance:
uuid: "9ad08b1c51af43d0bf6ae39a73efaf60"
format_stamp: "Formatted at 2026-09-29 10:37:35 on dist-test-slave-vg89"
server_key: "b77ab451f21f7fdea337398dfcba1c8b"
server_key_iv: "9fe7d5fad3743c00b5b9627808eae08d"
server_key_version: "encryptionkey@0"
I20260929 10:37:35.407696 5037 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260929 10:37:35.410215 5052 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:35.411506 5037 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260929 10:37:35.411659 5037 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790677981592896-15689-0/minicluster-data/master-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790677981592896-15689-0/minicluster-data/master-0/wal
uuid: "9ad08b1c51af43d0bf6ae39a73efaf60"
format_stamp: "Formatted at 2026-09-29 10:37:35 on dist-test-slave-vg89"
server_key: "b77ab451f21f7fdea337398dfcba1c8b"
server_key_iv: "9fe7d5fad3743c00b5b9627808eae08d"
server_key_version: "encryptionkey@0"
I20260929 10:37:35.411775 5037 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790677981592896-15689-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790677981592896-15689-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790677981592896-15689-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:37:35.416543 5037 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:37:35.419757 5037 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:37:35.419968 5037 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:37:35.427088 5037 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.126:34087
I20260929 10:37:35.427104 5104 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.126:34087 every 8 connection(s)
I20260929 10:37:35.428041 5037 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
I20260929 10:37:35.430836 5105 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:37:35.435806 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 5037
I20260929 10:37:35.435926 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790677981592896-15689-0/minicluster-data/master-0/wal/instance
I20260929 10:37:35.436229 15689 external_mini_cluster.cc:1428] Setting key 9d509e7bd83555f4891d13a7d69036a1
I20260929 10:37:35.436573 5105 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ad08b1c51af43d0bf6ae39a73efaf60: Bootstrap starting.
I20260929 10:37:35.439330 5105 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9ad08b1c51af43d0bf6ae39a73efaf60: Neither blocks nor log segments found. Creating new log.
I20260929 10:37:35.440229 5105 log.cc:824] T 00000000000000000000000000000000 P 9ad08b1c51af43d0bf6ae39a73efaf60: Log is configured to *not* fsync() on all Append() calls
I20260929 10:37:35.442286 5108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:35.437698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48357 (local address 127.15.82.126:34087)
0929 10:37:35.438146 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:35.438170 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:35.438214 (+ 44us) server_negotiation.cc:407] Connection header received
0929 10:37:35.439095 (+ 881us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:35.439118 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:35.439507 (+ 389us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:35.439895 (+ 388us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:35.440901 (+ 1006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:35.440906 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:35.440918 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:37:35.440928 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:35.440978 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:37:35.440998 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:35.441129 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:35.441270 (+ 141us) server_negotiation.cc:299] Negotiation successful
0929 10:37:35.441428 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":109,"threads_started":1}
I20260929 10:37:35.442575 5105 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ad08b1c51af43d0bf6ae39a73efaf60: No bootstrap required, opened a new log
I20260929 10:37:35.445256 15689 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 10:37:35.446637 5105 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9ad08b1c51af43d0bf6ae39a73efaf60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ad08b1c51af43d0bf6ae39a73efaf60" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 34087 } }
I20260929 10:37:35.446849 5105 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9ad08b1c51af43d0bf6ae39a73efaf60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:37:35.446924 5105 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9ad08b1c51af43d0bf6ae39a73efaf60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ad08b1c51af43d0bf6ae39a73efaf60, State: Initialized, Role: FOLLOWER
I20260929 10:37:35.447415 5105 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9ad08b1c51af43d0bf6ae39a73efaf60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ad08b1c51af43d0bf6ae39a73efaf60" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 34087 } }
I20260929 10:37:35.447582 5105 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9ad08b1c51af43d0bf6ae39a73efaf60 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:37:35.447647 5105 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9ad08b1c51af43d0bf6ae39a73efaf60 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:37:35.447744 5105 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9ad08b1c51af43d0bf6ae39a73efaf60 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:37:35.448650 5105 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9ad08b1c51af43d0bf6ae39a73efaf60 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ad08b1c51af43d0bf6ae39a73efaf60" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 34087 } }
I20260929 10:37:35.449010 5108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:35.447371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57933 (local address 127.15.82.126:34087)
0929 10:37:35.447491 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:35.447495 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:35.447643 (+ 148us) server_negotiation.cc:407] Connection header received
0929 10:37:35.447799 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:35.447803 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:35.447834 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:35.447906 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:35.448612 (+ 706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:35.448615 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:35.448617 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:37:35.448629 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:35.448655 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:37:35.448673 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:35.448762 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:35.448846 (+ 84us) server_negotiation.cc:299] Negotiation successful
0929 10:37:35.448885 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260929 10:37:35.449062 5105 leader_election.cc:302] T 00000000000000000000000000000000 P 9ad08b1c51af43d0bf6ae39a73efaf60 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ad08b1c51af43d0bf6ae39a73efaf60; no voters:
I20260929 10:37:35.449427 5105 leader_election.cc:288] T 00000000000000000000000000000000 P 9ad08b1c51af43d0bf6ae39a73efaf60 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:37:35.449499 5115 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9ad08b1c51af43d0bf6ae39a73efaf60 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:37:35.449695 5115 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9ad08b1c51af43d0bf6ae39a73efaf60 [term 1 LEADER]: Becoming Leader. State: Replica: 9ad08b1c51af43d0bf6ae39a73efaf60, State: Running, Role: LEADER
I20260929 10:37:35.449988 5115 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9ad08b1c51af43d0bf6ae39a73efaf60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ad08b1c51af43d0bf6ae39a73efaf60" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 34087 } }
I20260929 10:37:35.450770 5105 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9ad08b1c51af43d0bf6ae39a73efaf60 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:37:35.451737 5116 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ad08b1c51af43d0bf6ae39a73efaf60 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9ad08b1c51af43d0bf6ae39a73efaf60. Latest consensus state: current_term: 1 leader_uuid: "9ad08b1c51af43d0bf6ae39a73efaf60" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ad08b1c51af43d0bf6ae39a73efaf60" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 34087 } } }
I20260929 10:37:35.451879 5116 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ad08b1c51af43d0bf6ae39a73efaf60 [sys.catalog]: This master's current role is: LEADER
I20260929 10:37:35.452158 5117 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ad08b1c51af43d0bf6ae39a73efaf60 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9ad08b1c51af43d0bf6ae39a73efaf60" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ad08b1c51af43d0bf6ae39a73efaf60" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 34087 } } }
I20260929 10:37:35.452220 5124 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:37:35.452248 5117 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ad08b1c51af43d0bf6ae39a73efaf60 [sys.catalog]: This master's current role is: LEADER
I20260929 10:37:35.455732 5124 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:37:35.460908 5124 catalog_manager.cc:1347] Generated new cluster ID: 4b98f9852cad46aa9fe1e638761ccb06
I20260929 10:37:35.460973 5124 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:37:35.479847 5124 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:37:35.480224 5124 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:37:35.488628 5124 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9ad08b1c51af43d0bf6ae39a73efaf60: Generated new TSK 0
I20260929 10:37:35.489498 5124 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 10:37:35.494606 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 5037
2026-09-29T10:37:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1447 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790677981592896-15689-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:37:35 dist-test-slave-vg89 krb5kdc[5136](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:37:35 dist-test-slave-vg89 krb5kdc[5136](info): set up 2 sockets
Sep 29 10:37:35 dist-test-slave-vg89 krb5kdc[5136](info): commencing operation
krb5kdc: starting...
W20260929 10:37:36.623044 15689 mini_kdc.cc:121] Time spent starting KDC: real 1.108s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Sep 29 10:37:36 dist-test-slave-vg89 krb5kdc[5136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678256, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:37:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:37:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.82.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab.
Entry for principal kudu/127.15.82.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab.
I20260929 10:37:36.757797 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790677981592896-15689-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790677981592896-15689-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790677981592896-15689-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:35793
--webserver_interface=127.15.82.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab
--principal=kudu/127.15.82.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:45077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.82.126:35793
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:37:36.873991 5149 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:37:36.874281 5149 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:37:36.874387 5149 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:37:36.878454 5149 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:37:36.878547 5149 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:37:36.878571 5149 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:37:36.878590 5149 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:37:36.878647 5149 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:37:36.883942 5149 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:45077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790677981592896-15689-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790677981592896-15689-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.82.126:35793
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab
--principal=kudu/127.15.82.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:35793
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.82.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5149
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790677981592896-15689-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:37:36.884990 5149 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:37:36.886045 5149 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:37:36.894649 5157 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:36.894712 5154 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:36.894649 5155 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:37:36.894899 5149 server_base.cc:1034] running on GCE node
I20260929 10:37:36.895543 5149 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:37:36.896648 5149 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:37:36.897842 5149 hybrid_clock.cc:648] HybridClock initialized: now 1790678256897827 us; error 26 us; skew 500 ppm
Sep 29 10:37:36 dist-test-slave-vg89 krb5kdc[5136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678256, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:37:36.906724 5149 init.cc:377] Logged in from keytab as kudu/127.15.82.126@KRBTEST.COM (short username kudu)
I20260929 10:37:36.907661 5149 webserver.cc:460] Webserver started at http://127.15.82.126:42865/ using document root <none> and password file <none>
I20260929 10:37:36.908234 5149 fs_manager.cc:359] Metadata directory not provided
I20260929 10:37:36.908308 5149 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:37:36.908519 5149 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:37:36.910692 5149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790677981592896-15689-0/minicluster-data/master-0/data/instance:
uuid: "01013777c4cf4783ac7c613d9d00cba1"
format_stamp: "Formatted at 2026-09-29 10:37:36 on dist-test-slave-vg89"
server_key: "85873a3c175e23f27c693f9eb1db2c70"
server_key_iv: "2a554b906de23a8a492b2eb82b5836c6"
server_key_version: "encryptionkey@0"
I20260929 10:37:36.911263 5149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790677981592896-15689-0/minicluster-data/master-0/wal/instance:
uuid: "01013777c4cf4783ac7c613d9d00cba1"
format_stamp: "Formatted at 2026-09-29 10:37:36 on dist-test-slave-vg89"
server_key: "85873a3c175e23f27c693f9eb1db2c70"
server_key_iv: "2a554b906de23a8a492b2eb82b5836c6"
server_key_version: "encryptionkey@0"
I20260929 10:37:36.915047 5149 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260929 10:37:36.917542 5164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:36.918985 5149 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260929 10:37:36.919130 5149 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790677981592896-15689-0/minicluster-data/master-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790677981592896-15689-0/minicluster-data/master-0/wal
uuid: "01013777c4cf4783ac7c613d9d00cba1"
format_stamp: "Formatted at 2026-09-29 10:37:36 on dist-test-slave-vg89"
server_key: "85873a3c175e23f27c693f9eb1db2c70"
server_key_iv: "2a554b906de23a8a492b2eb82b5836c6"
server_key_version: "encryptionkey@0"
I20260929 10:37:36.919242 5149 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790677981592896-15689-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790677981592896-15689-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790677981592896-15689-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:37:36.939426 5149 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:37:36.942965 5149 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:37:36.943183 5149 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:37:36.950191 5149 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.126:35793
I20260929 10:37:36.950277 5216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.126:35793 every 8 connection(s)
I20260929 10:37:36.951210 5149 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
I20260929 10:37:36.953974 5217 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:37:36.955142 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 5149
I20260929 10:37:36.955345 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790677981592896-15689-0/minicluster-data/master-0/wal/instance
I20260929 10:37:36.955683 15689 external_mini_cluster.cc:1428] Setting key afad10163d7409d8564315b49bf1065a
I20260929 10:37:36.960938 5217 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01013777c4cf4783ac7c613d9d00cba1: Bootstrap starting.
I20260929 10:37:36.963515 5217 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 01013777c4cf4783ac7c613d9d00cba1: Neither blocks nor log segments found. Creating new log.
I20260929 10:37:36.964265 5217 log.cc:824] T 00000000000000000000000000000000 P 01013777c4cf4783ac7c613d9d00cba1: Log is configured to *not* fsync() on all Append() calls
Sep 29 10:37:36 dist-test-slave-vg89 krb5kdc[5136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678256, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:37:36.966285 5217 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01013777c4cf4783ac7c613d9d00cba1: No bootstrap required, opened a new log
I20260929 10:37:36.971170 5220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:36.957084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33915 (local address 127.15.82.126:35793)
0929 10:37:36.957497 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:36.957517 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:36.957563 (+ 46us) server_negotiation.cc:407] Connection header received
0929 10:37:36.958379 (+ 816us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:36.958429 (+ 50us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:36.958864 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:36.959223 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:36.959910 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:36.961325 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:36.962159 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:36.962844 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:36.962998 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:36.965754 (+ 2756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:36.965760 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:36.965777 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:37:36.965806 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:36.967878 (+ 2072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:36.968328 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:36.968333 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:36.968341 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:36.968493 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:36.968768 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:36.968772 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:36.968775 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:36.969328 (+ 553us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:36.969506 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:36.969857 (+ 351us) server_negotiation.cc:299] Negotiation successful
0929 10:37:36.970159 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":110,"threads_started":1}
I20260929 10:37:36.971038 5217 raft_consensus.cc:348] T 00000000000000000000000000000000 P 01013777c4cf4783ac7c613d9d00cba1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01013777c4cf4783ac7c613d9d00cba1" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 35793 } }
I20260929 10:37:36.971293 5217 raft_consensus.cc:374] T 00000000000000000000000000000000 P 01013777c4cf4783ac7c613d9d00cba1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:37:36.971346 5217 raft_consensus.cc:729] T 00000000000000000000000000000000 P 01013777c4cf4783ac7c613d9d00cba1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01013777c4cf4783ac7c613d9d00cba1, State: Initialized, Role: FOLLOWER
I20260929 10:37:36.971807 5217 consensus_queue.cc:260] T 00000000000000000000000000000000 P 01013777c4cf4783ac7c613d9d00cba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01013777c4cf4783ac7c613d9d00cba1" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 35793 } }
I20260929 10:37:36.971961 5217 raft_consensus.cc:388] T 00000000000000000000000000000000 P 01013777c4cf4783ac7c613d9d00cba1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:37:36.972014 5217 raft_consensus.cc:482] T 00000000000000000000000000000000 P 01013777c4cf4783ac7c613d9d00cba1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:37:36.972114 5217 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 01013777c4cf4783ac7c613d9d00cba1 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:37:36.973076 5217 raft_consensus.cc:504] T 00000000000000000000000000000000 P 01013777c4cf4783ac7c613d9d00cba1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01013777c4cf4783ac7c613d9d00cba1" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 35793 } }
I20260929 10:37:36.973397 5217 leader_election.cc:302] T 00000000000000000000000000000000 P 01013777c4cf4783ac7c613d9d00cba1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 01013777c4cf4783ac7c613d9d00cba1; no voters:
I20260929 10:37:36.973739 5217 leader_election.cc:288] T 00000000000000000000000000000000 P 01013777c4cf4783ac7c613d9d00cba1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:37:36.973861 5222 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 01013777c4cf4783ac7c613d9d00cba1 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:37:36.974081 15689 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 10:37:36.974079 5222 raft_consensus.cc:686] T 00000000000000000000000000000000 P 01013777c4cf4783ac7c613d9d00cba1 [term 1 LEADER]: Becoming Leader. State: Replica: 01013777c4cf4783ac7c613d9d00cba1, State: Running, Role: LEADER
I20260929 10:37:36.974427 5222 consensus_queue.cc:237] T 00000000000000000000000000000000 P 01013777c4cf4783ac7c613d9d00cba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01013777c4cf4783ac7c613d9d00cba1" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 35793 } }
I20260929 10:37:36.975039 5217 sys_catalog.cc:564] T 00000000000000000000000000000000 P 01013777c4cf4783ac7c613d9d00cba1 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:37:36.976106 5223 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01013777c4cf4783ac7c613d9d00cba1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "01013777c4cf4783ac7c613d9d00cba1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01013777c4cf4783ac7c613d9d00cba1" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 35793 } } }
I20260929 10:37:36.976231 5223 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01013777c4cf4783ac7c613d9d00cba1 [sys.catalog]: This master's current role is: LEADER
I20260929 10:37:36.976590 5224 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01013777c4cf4783ac7c613d9d00cba1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01013777c4cf4783ac7c613d9d00cba1. Latest consensus state: current_term: 1 leader_uuid: "01013777c4cf4783ac7c613d9d00cba1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01013777c4cf4783ac7c613d9d00cba1" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 35793 } } }
I20260929 10:37:36.976706 5224 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01013777c4cf4783ac7c613d9d00cba1 [sys.catalog]: This master's current role is: LEADER
I20260929 10:37:36.976670 5235 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:37:36.980438 5235 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:37:36.986295 5220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:36.977453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54081 (local address 127.15.82.126:35793)
0929 10:37:36.977567 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:36.977572 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:36.977799 (+ 227us) server_negotiation.cc:407] Connection header received
0929 10:37:36.977951 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:36.977954 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:36.978015 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:36.978086 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:36.978975 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:36.979737 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:36.980883 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:36.981196 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:36.981273 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:36.982991 (+ 1718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:36.982995 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:36.982997 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:37:36.983027 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:36.984479 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:36.985003 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:36.985005 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:36.985006 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:36.985074 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:36.985377 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:36.985380 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:36.985382 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:36.985603 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:36.985749 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:36.986006 (+ 257us) server_negotiation.cc:299] Negotiation successful
0929 10:37:36.986127 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260929 10:37:36.986900 5235 catalog_manager.cc:1347] Generated new cluster ID: 179006aa33a94f69abe408c89b2b1a5c
I20260929 10:37:36.986963 5235 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:37:36.996762 5235 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:37:36.997974 5235 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:37:37.004518 5235 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 01013777c4cf4783ac7c613d9d00cba1: Generated new TSK 0
I20260929 10:37:37.005155 5235 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 10:37:37.013731 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 5149
2026-09-29T10:37:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1518 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790677981592896-15689-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:37:37 dist-test-slave-vg89 krb5kdc[5248](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:37:37 dist-test-slave-vg89 krb5kdc[5248](info): set up 2 sockets
Sep 29 10:37:37 dist-test-slave-vg89 krb5kdc[5248](info): commencing operation
krb5kdc: starting...
W20260929 10:37:38.059031 15689 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Sep 29 10:37:38 dist-test-slave-vg89 krb5kdc[5248](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678258, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:37:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:37:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.82.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab.
Entry for principal kudu/127.15.82.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab.
I20260929 10:37:38.189633 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790677981592896-15689-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790677981592896-15689-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790677981592896-15689-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:40813
--webserver_interface=127.15.82.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab
--principal=kudu/127.15.82.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:34939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.82.126:40813
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:37:38.302386 5261 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:37:38.302732 5261 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:37:38.302810 5261 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:37:38.306862 5261 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:37:38.306965 5261 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:37:38.307003 5261 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:37:38.307040 5261 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:37:38.307071 5261 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:37:38.313782 5261 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:34939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790677981592896-15689-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790677981592896-15689-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.82.126:40813
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab
--principal=kudu/127.15.82.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:40813
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.82.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5261
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790677981592896-15689-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:37:38.315057 5261 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:37:38.316154 5261 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:37:38.325227 5269 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:38.325227 5266 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:37:38.325409 5261 server_base.cc:1034] running on GCE node
W20260929 10:37:38.325284 5267 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:37:38.325992 5261 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:37:38.327242 5261 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:37:38.328470 5261 hybrid_clock.cc:648] HybridClock initialized: now 1790678258328450 us; error 31 us; skew 500 ppm
Sep 29 10:37:38 dist-test-slave-vg89 krb5kdc[5248](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678258, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:37:38.331490 5261 init.cc:377] Logged in from keytab as kudu/127.15.82.126@KRBTEST.COM (short username kudu)
I20260929 10:37:38.332456 5261 webserver.cc:460] Webserver started at http://127.15.82.126:45707/ using document root <none> and password file <none>
I20260929 10:37:38.333003 5261 fs_manager.cc:359] Metadata directory not provided
I20260929 10:37:38.333063 5261 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:37:38.333236 5261 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:37:38.335428 5261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790677981592896-15689-0/minicluster-data/master-0/data/instance:
uuid: "6d9da17f65494d0dae89b2a0d62a205c"
format_stamp: "Formatted at 2026-09-29 10:37:38 on dist-test-slave-vg89"
server_key: "e09c53b330a45bf0e3a12b1be0e89803"
server_key_iv: "b0f2cdfe7008d88631eac066c9eac47b"
server_key_version: "encryptionkey@0"
I20260929 10:37:38.335963 5261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790677981592896-15689-0/minicluster-data/master-0/wal/instance:
uuid: "6d9da17f65494d0dae89b2a0d62a205c"
format_stamp: "Formatted at 2026-09-29 10:37:38 on dist-test-slave-vg89"
server_key: "e09c53b330a45bf0e3a12b1be0e89803"
server_key_iv: "b0f2cdfe7008d88631eac066c9eac47b"
server_key_version: "encryptionkey@0"
I20260929 10:37:38.339934 5261 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 10:37:38.342602 5276 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:38.343979 5261 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 10:37:38.344103 5261 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790677981592896-15689-0/minicluster-data/master-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790677981592896-15689-0/minicluster-data/master-0/wal
uuid: "6d9da17f65494d0dae89b2a0d62a205c"
format_stamp: "Formatted at 2026-09-29 10:37:38 on dist-test-slave-vg89"
server_key: "e09c53b330a45bf0e3a12b1be0e89803"
server_key_iv: "b0f2cdfe7008d88631eac066c9eac47b"
server_key_version: "encryptionkey@0"
I20260929 10:37:38.344208 5261 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790677981592896-15689-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790677981592896-15689-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790677981592896-15689-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:37:38.381414 5261 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:37:38.384754 5261 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:37:38.384930 5261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:37:38.392189 5261 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.126:40813
I20260929 10:37:38.392204 5328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.126:40813 every 8 connection(s)
I20260929 10:37:38.393208 5261 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
I20260929 10:37:38.396100 5329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:37:38.397168 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 5261
I20260929 10:37:38.397305 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790677981592896-15689-0/minicluster-data/master-0/wal/instance
I20260929 10:37:38.397639 15689 external_mini_cluster.cc:1428] Setting key cab679991a8e71dac98b0131cac2b229
I20260929 10:37:38.402721 5329 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d9da17f65494d0dae89b2a0d62a205c: Bootstrap starting.
I20260929 10:37:38.405424 5329 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6d9da17f65494d0dae89b2a0d62a205c: Neither blocks nor log segments found. Creating new log.
I20260929 10:37:38.406297 5329 log.cc:824] T 00000000000000000000000000000000 P 6d9da17f65494d0dae89b2a0d62a205c: Log is configured to *not* fsync() on all Append() calls
I20260929 10:37:38.407114 5332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:38.399180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36581 (local address 127.15.82.126:40813)
0929 10:37:38.399605 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:38.399621 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:38.399657 (+ 36us) server_negotiation.cc:407] Connection header received
0929 10:37:38.400309 (+ 652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:38.400325 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:38.400667 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:38.400959 (+ 292us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:38.401344 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:38.402350 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:38.403641 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:38.404116 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:38.404236 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:38.405158 (+ 922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:38.405163 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:38.405176 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:37:38.405624 (+ 448us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:38.405678 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:37:38.405701 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:38.405775 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:38.405939 (+ 164us) server_negotiation.cc:299] Negotiation successful
0929 10:37:38.406173 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":150,"threads_started":1}
I20260929 10:37:38.408394 5329 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d9da17f65494d0dae89b2a0d62a205c: No bootstrap required, opened a new log
I20260929 10:37:38.410030 15689 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 10:37:38.412639 5329 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6d9da17f65494d0dae89b2a0d62a205c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d9da17f65494d0dae89b2a0d62a205c" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 40813 } }
I20260929 10:37:38.412976 5329 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6d9da17f65494d0dae89b2a0d62a205c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:37:38.413075 5329 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6d9da17f65494d0dae89b2a0d62a205c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d9da17f65494d0dae89b2a0d62a205c, State: Initialized, Role: FOLLOWER
I20260929 10:37:38.413542 5329 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d9da17f65494d0dae89b2a0d62a205c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d9da17f65494d0dae89b2a0d62a205c" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 40813 } }
I20260929 10:37:38.413695 5329 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6d9da17f65494d0dae89b2a0d62a205c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:37:38.413781 5329 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6d9da17f65494d0dae89b2a0d62a205c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:37:38.413885 5329 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6d9da17f65494d0dae89b2a0d62a205c [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:37:38.415082 5329 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6d9da17f65494d0dae89b2a0d62a205c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d9da17f65494d0dae89b2a0d62a205c" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 40813 } }
I20260929 10:37:38.415467 5329 leader_election.cc:302] T 00000000000000000000000000000000 P 6d9da17f65494d0dae89b2a0d62a205c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6d9da17f65494d0dae89b2a0d62a205c; no voters:
I20260929 10:37:38.415793 5329 leader_election.cc:288] T 00000000000000000000000000000000 P 6d9da17f65494d0dae89b2a0d62a205c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:37:38.415895 5339 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6d9da17f65494d0dae89b2a0d62a205c [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:37:38.416158 5339 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6d9da17f65494d0dae89b2a0d62a205c [term 1 LEADER]: Becoming Leader. State: Replica: 6d9da17f65494d0dae89b2a0d62a205c, State: Running, Role: LEADER
I20260929 10:37:38.416508 5339 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d9da17f65494d0dae89b2a0d62a205c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d9da17f65494d0dae89b2a0d62a205c" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 40813 } }
I20260929 10:37:38.416658 5332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:38.412129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51459 (local address 127.15.82.126:40813)
0929 10:37:38.412259 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:38.412264 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:38.412456 (+ 192us) server_negotiation.cc:407] Connection header received
0929 10:37:38.412579 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:38.412581 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:38.412620 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:38.412677 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:38.413000 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:38.413592 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:38.414736 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:38.414988 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:38.415078 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:38.416024 (+ 946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:38.416029 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:38.416032 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:37:38.416046 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:38.416075 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:37:38.416088 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:38.416166 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:38.416458 (+ 292us) server_negotiation.cc:299] Negotiation successful
0929 10:37:38.416512 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260929 10:37:38.417205 5329 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6d9da17f65494d0dae89b2a0d62a205c [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:37:38.417744 5340 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d9da17f65494d0dae89b2a0d62a205c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6d9da17f65494d0dae89b2a0d62a205c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d9da17f65494d0dae89b2a0d62a205c" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 40813 } } }
I20260929 10:37:38.417876 5340 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d9da17f65494d0dae89b2a0d62a205c [sys.catalog]: This master's current role is: LEADER
I20260929 10:37:38.418695 5345 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:37:38.418643 5341 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d9da17f65494d0dae89b2a0d62a205c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d9da17f65494d0dae89b2a0d62a205c. Latest consensus state: current_term: 1 leader_uuid: "6d9da17f65494d0dae89b2a0d62a205c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d9da17f65494d0dae89b2a0d62a205c" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 40813 } } }
I20260929 10:37:38.418830 5341 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d9da17f65494d0dae89b2a0d62a205c [sys.catalog]: This master's current role is: LEADER
I20260929 10:37:38.421742 5345 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:37:38.427309 5345 catalog_manager.cc:1347] Generated new cluster ID: 9f4fa5e664024fe984cf7a8bb3a5fb6a
I20260929 10:37:38.427380 5345 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:37:38.446697 5345 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:37:38.448159 5345 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:37:38.457789 5345 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6d9da17f65494d0dae89b2a0d62a205c: Generated new TSK 0
I20260929 10:37:38.458736 5345 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 10:37:38.469694 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 5261
2026-09-29T10:37:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1455 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790677981592896-15689-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:37:38 dist-test-slave-vg89 krb5kdc[5360](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:37:38 dist-test-slave-vg89 krb5kdc[5360](info): set up 2 sockets
Sep 29 10:37:38 dist-test-slave-vg89 krb5kdc[5360](info): commencing operation
krb5kdc: starting...
W20260929 10:37:39.499360 15689 mini_kdc.cc:121] Time spent starting KDC: real 1.010s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Sep 29 10:37:39 dist-test-slave-vg89 krb5kdc[5360](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678259, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:37:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:37:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.82.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab.
Entry for principal kudu/127.15.82.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab.
I20260929 10:37:39.629309 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790677981592896-15689-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790677981592896-15689-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790677981592896-15689-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:46709
--webserver_interface=127.15.82.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab
--principal=kudu/127.15.82.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:46457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.82.126:46709
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:37:39.739549 5373 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:37:39.739843 5373 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:37:39.739888 5373 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:37:39.743940 5373 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:37:39.744009 5373 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:37:39.744028 5373 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:37:39.744045 5373 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:37:39.744060 5373 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:37:39.749300 5373 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:46457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790677981592896-15689-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790677981592896-15689-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.82.126:46709
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab
--principal=kudu/127.15.82.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:46709
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.82.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5373
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790677981592896-15689-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:37:39.750312 5373 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:37:39.751376 5373 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:37:39.759763 5378 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:39.759786 5379 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:39.759805 5381 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:37:39.760043 5373 server_base.cc:1034] running on GCE node
I20260929 10:37:39.760665 5373 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:37:39.761824 5373 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:37:39.763019 5373 hybrid_clock.cc:648] HybridClock initialized: now 1790678259762986 us; error 43 us; skew 500 ppm
Sep 29 10:37:39 dist-test-slave-vg89 krb5kdc[5360](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678259, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:37:39.765944 5373 init.cc:377] Logged in from keytab as kudu/127.15.82.126@KRBTEST.COM (short username kudu)
I20260929 10:37:39.766908 5373 webserver.cc:460] Webserver started at http://127.15.82.126:33389/ using document root <none> and password file <none>
I20260929 10:37:39.767452 5373 fs_manager.cc:359] Metadata directory not provided
I20260929 10:37:39.767511 5373 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:37:39.767728 5373 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:37:39.769691 5373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790677981592896-15689-0/minicluster-data/master-0/data/instance:
uuid: "359def1f6c5b42e8b63f29cefbebff2a"
format_stamp: "Formatted at 2026-09-29 10:37:39 on dist-test-slave-vg89"
server_key: "3a1b393c21281df33bd84e0c8d14dfce"
server_key_iv: "712a8940d1795831e49ea12037ce1554"
server_key_version: "encryptionkey@0"
I20260929 10:37:39.770258 5373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790677981592896-15689-0/minicluster-data/master-0/wal/instance:
uuid: "359def1f6c5b42e8b63f29cefbebff2a"
format_stamp: "Formatted at 2026-09-29 10:37:39 on dist-test-slave-vg89"
server_key: "3a1b393c21281df33bd84e0c8d14dfce"
server_key_iv: "712a8940d1795831e49ea12037ce1554"
server_key_version: "encryptionkey@0"
I20260929 10:37:39.773949 5373 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260929 10:37:39.776554 5388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:39.777884 5373 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 10:37:39.777994 5373 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790677981592896-15689-0/minicluster-data/master-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790677981592896-15689-0/minicluster-data/master-0/wal
uuid: "359def1f6c5b42e8b63f29cefbebff2a"
format_stamp: "Formatted at 2026-09-29 10:37:39 on dist-test-slave-vg89"
server_key: "3a1b393c21281df33bd84e0c8d14dfce"
server_key_iv: "712a8940d1795831e49ea12037ce1554"
server_key_version: "encryptionkey@0"
I20260929 10:37:39.778136 5373 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790677981592896-15689-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790677981592896-15689-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790677981592896-15689-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:37:39.783027 5373 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:37:39.786324 5373 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:37:39.786595 5373 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:37:39.793942 5373 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.126:46709
I20260929 10:37:39.793968 5440 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.126:46709 every 8 connection(s)
I20260929 10:37:39.794981 5373 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
I20260929 10:37:39.795538 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 5373
I20260929 10:37:39.795707 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790677981592896-15689-0/minicluster-data/master-0/wal/instance
I20260929 10:37:39.796075 15689 external_mini_cluster.cc:1428] Setting key 103113160b0237d911f26426a73ef5e4
I20260929 10:37:39.798022 5441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:37:39.801676 5444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:39.797531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43373 (local address 127.15.82.126:46709)
0929 10:37:39.797952 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:39.797968 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:39.798006 (+ 38us) server_negotiation.cc:407] Connection header received
0929 10:37:39.798781 (+ 775us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:39.798800 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:39.799113 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:39.799478 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:39.800312 (+ 834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:39.800317 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:39.800330 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:37:39.800341 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:39.800396 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:37:39.800415 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:39.800573 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:39.800646 (+ 73us) server_negotiation.cc:299] Negotiation successful
0929 10:37:39.800941 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":143,"threads_started":1}
I20260929 10:37:39.804616 5441 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 359def1f6c5b42e8b63f29cefbebff2a: Bootstrap starting.
I20260929 10:37:39.804962 15689 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 10:37:39.807165 5441 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 359def1f6c5b42e8b63f29cefbebff2a: Neither blocks nor log segments found. Creating new log.
I20260929 10:37:39.808007 5441 log.cc:824] T 00000000000000000000000000000000 P 359def1f6c5b42e8b63f29cefbebff2a: Log is configured to *not* fsync() on all Append() calls
I20260929 10:37:39.809165 5444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:39.807251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56667 (local address 127.15.82.126:46709)
0929 10:37:39.807388 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:39.807393 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:39.807554 (+ 161us) server_negotiation.cc:407] Connection header received
0929 10:37:39.807702 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:39.807705 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:39.807783 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:39.807875 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:39.808694 (+ 819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:39.808697 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:39.808699 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:37:39.808709 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:39.808734 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:37:39.808744 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:39.808865 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:39.809008 (+ 143us) server_negotiation.cc:299] Negotiation successful
0929 10:37:39.809052 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260929 10:37:39.810446 5441 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 359def1f6c5b42e8b63f29cefbebff2a: No bootstrap required, opened a new log
I20260929 10:37:39.813915 5441 raft_consensus.cc:348] T 00000000000000000000000000000000 P 359def1f6c5b42e8b63f29cefbebff2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "359def1f6c5b42e8b63f29cefbebff2a" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 46709 } }
I20260929 10:37:39.814142 5441 raft_consensus.cc:374] T 00000000000000000000000000000000 P 359def1f6c5b42e8b63f29cefbebff2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:37:39.814177 5441 raft_consensus.cc:729] T 00000000000000000000000000000000 P 359def1f6c5b42e8b63f29cefbebff2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 359def1f6c5b42e8b63f29cefbebff2a, State: Initialized, Role: FOLLOWER
I20260929 10:37:39.814635 5441 consensus_queue.cc:260] T 00000000000000000000000000000000 P 359def1f6c5b42e8b63f29cefbebff2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "359def1f6c5b42e8b63f29cefbebff2a" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 46709 } }
I20260929 10:37:39.814761 5441 raft_consensus.cc:388] T 00000000000000000000000000000000 P 359def1f6c5b42e8b63f29cefbebff2a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:37:39.814831 5441 raft_consensus.cc:482] T 00000000000000000000000000000000 P 359def1f6c5b42e8b63f29cefbebff2a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:37:39.814954 5441 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 359def1f6c5b42e8b63f29cefbebff2a [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:37:39.815888 5441 raft_consensus.cc:504] T 00000000000000000000000000000000 P 359def1f6c5b42e8b63f29cefbebff2a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "359def1f6c5b42e8b63f29cefbebff2a" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 46709 } }
I20260929 10:37:39.816241 5441 leader_election.cc:302] T 00000000000000000000000000000000 P 359def1f6c5b42e8b63f29cefbebff2a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 359def1f6c5b42e8b63f29cefbebff2a; no voters:
I20260929 10:37:39.816550 5441 leader_election.cc:288] T 00000000000000000000000000000000 P 359def1f6c5b42e8b63f29cefbebff2a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:37:39.816779 5451 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 359def1f6c5b42e8b63f29cefbebff2a [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:37:39.817072 5451 raft_consensus.cc:686] T 00000000000000000000000000000000 P 359def1f6c5b42e8b63f29cefbebff2a [term 1 LEADER]: Becoming Leader. State: Replica: 359def1f6c5b42e8b63f29cefbebff2a, State: Running, Role: LEADER
I20260929 10:37:39.817458 5451 consensus_queue.cc:237] T 00000000000000000000000000000000 P 359def1f6c5b42e8b63f29cefbebff2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "359def1f6c5b42e8b63f29cefbebff2a" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 46709 } }
I20260929 10:37:39.817865 5441 sys_catalog.cc:564] T 00000000000000000000000000000000 P 359def1f6c5b42e8b63f29cefbebff2a [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:37:39.819428 5453 sys_catalog.cc:455] T 00000000000000000000000000000000 P 359def1f6c5b42e8b63f29cefbebff2a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 359def1f6c5b42e8b63f29cefbebff2a. Latest consensus state: current_term: 1 leader_uuid: "359def1f6c5b42e8b63f29cefbebff2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "359def1f6c5b42e8b63f29cefbebff2a" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 46709 } } }
I20260929 10:37:39.819581 5453 sys_catalog.cc:458] T 00000000000000000000000000000000 P 359def1f6c5b42e8b63f29cefbebff2a [sys.catalog]: This master's current role is: LEADER
I20260929 10:37:39.819902 5452 sys_catalog.cc:455] T 00000000000000000000000000000000 P 359def1f6c5b42e8b63f29cefbebff2a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "359def1f6c5b42e8b63f29cefbebff2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "359def1f6c5b42e8b63f29cefbebff2a" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 46709 } } }
I20260929 10:37:39.819991 5452 sys_catalog.cc:458] T 00000000000000000000000000000000 P 359def1f6c5b42e8b63f29cefbebff2a [sys.catalog]: This master's current role is: LEADER
W20260929 10:37:39.823215 5466 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 359def1f6c5b42e8b63f29cefbebff2a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 10:37:39.823280 5466 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 10:37:39.823364 5463 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:37:39.824043 5463 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:37:39.829298 5463 catalog_manager.cc:1347] Generated new cluster ID: 5a1d2f40144f4d58b211fe19a6e2ca5a
I20260929 10:37:39.829370 5463 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:37:39.854933 5463 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:37:39.855297 5463 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:37:39.862015 5463 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 359def1f6c5b42e8b63f29cefbebff2a: Generated new TSK 0
I20260929 10:37:39.862848 5463 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 10:37:39.866968 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 5373
2026-09-29T10:37:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1398 ms)
[----------] 18 tests from AuthTokenIssuingTest (25729 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790677981592896-15689-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:37:39 dist-test-slave-vg89 krb5kdc[5472](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:37:39 dist-test-slave-vg89 krb5kdc[5472](info): set up 2 sockets
Sep 29 10:37:39 dist-test-slave-vg89 krb5kdc[5472](info): commencing operation
krb5kdc: starting...
W20260929 10:37:40.907073 15689 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Sep 29 10:37:40 dist-test-slave-vg89 krb5kdc[5472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678260, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:37:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:37:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.82.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab.
Entry for principal kudu/127.15.82.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab.
I20260929 10:37:41.036936 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790677981592896-15689-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790677981592896-15689-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790677981592896-15689-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:38805
--webserver_interface=127.15.82.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab
--principal=kudu/127.15.82.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:39801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.82.126:38805,127.15.82.125:42085,127.15.82.124:42611
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:37:41.151746 5485 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 10:37:41.152138 5485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:37:41.152194 5485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:37:41.152287 5485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:37:41.156350 5485 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:37:41.156453 5485 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:37:41.156493 5485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:37:41.156531 5485 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:37:41.156567 5485 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:37:41.161904 5485 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:39801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790677981592896-15689-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790677981592896-15689-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.82.126:38805,127.15.82.125:42085,127.15.82.124:42611
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab
--principal=kudu/127.15.82.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:38805
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.82.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5485
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790677981592896-15689-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:37:41.163111 5485 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:37:41.164211 5485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 10:37:41.173681 5485 server_base.cc:1034] running on GCE node
W20260929 10:37:41.173632 5493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:41.173615 5490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:41.173611 5491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:37:41.174695 5485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:37:41.175812 5485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:37:41.176982 5485 hybrid_clock.cc:648] HybridClock initialized: now 1790678261176969 us; error 28 us; skew 500 ppm
Sep 29 10:37:41 dist-test-slave-vg89 krb5kdc[5472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678261, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:37:41.179899 5485 init.cc:377] Logged in from keytab as kudu/127.15.82.126@KRBTEST.COM (short username kudu)
I20260929 10:37:41.180888 5485 webserver.cc:460] Webserver started at http://127.15.82.126:35991/ using document root <none> and password file <none>
I20260929 10:37:41.181479 5485 fs_manager.cc:359] Metadata directory not provided
I20260929 10:37:41.181540 5485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:37:41.181758 5485 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:37:41.183925 5485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790677981592896-15689-0/minicluster-data/master-0/data/instance:
uuid: "5173f7f7114d495a8774942645187c35"
format_stamp: "Formatted at 2026-09-29 10:37:41 on dist-test-slave-vg89"
server_key: "4465ff6dadbc4fd761fe9002839b89bc"
server_key_iv: "fd66c45213085d870fc607800626db40"
server_key_version: "encryptionkey@0"
I20260929 10:37:41.184574 5485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790677981592896-15689-0/minicluster-data/master-0/wal/instance:
uuid: "5173f7f7114d495a8774942645187c35"
format_stamp: "Formatted at 2026-09-29 10:37:41 on dist-test-slave-vg89"
server_key: "4465ff6dadbc4fd761fe9002839b89bc"
server_key_iv: "fd66c45213085d870fc607800626db40"
server_key_version: "encryptionkey@0"
I20260929 10:37:41.188843 5485 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 10:37:41.191370 5500 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:41.192633 5485 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 10:37:41.192775 5485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790677981592896-15689-0/minicluster-data/master-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790677981592896-15689-0/minicluster-data/master-0/wal
uuid: "5173f7f7114d495a8774942645187c35"
format_stamp: "Formatted at 2026-09-29 10:37:41 on dist-test-slave-vg89"
server_key: "4465ff6dadbc4fd761fe9002839b89bc"
server_key_iv: "fd66c45213085d870fc607800626db40"
server_key_version: "encryptionkey@0"
I20260929 10:37:41.192890 5485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790677981592896-15689-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790677981592896-15689-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790677981592896-15689-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:37:41.215098 5485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:37:41.218442 5485 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:37:41.218679 5485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:37:41.225960 5485 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.126:38805
I20260929 10:37:41.226318 5552 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.126:38805 every 8 connection(s)
I20260929 10:37:41.227010 5485 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
I20260929 10:37:41.229820 5553 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:37:41.233613 5553 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.82.126" port: 38805 } has no permanent_uuid. Determining permanent_uuid...
I20260929 10:37:41.234624 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 5485
I20260929 10:37:41.234753 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790677981592896-15689-0/minicluster-data/master-0/wal/instance
I20260929 10:37:41.235034 15689 external_mini_cluster.cc:1428] Setting key 6e4fd547879665fd4bd4ba28a9b1a396
WARNING: no policy specified for kudu/127.15.82.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.125@KRBTEST.COM" created.
Sep 29 10:37:41 dist-test-slave-vg89 krb5kdc[5472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678261, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.126@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:37:41.253228 5555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:41.236266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44694 (local address 127.15.82.126:38805)
0929 10:37:41.236821 (+ 555us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:41.236842 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:41.237097 (+ 255us) server_negotiation.cc:407] Connection header received
0929 10:37:41.238138 (+ 1041us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:41.238146 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:41.238385 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:41.238640 (+ 255us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:41.239473 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:41.241034 (+ 1561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:41.242077 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:41.242747 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:41.242949 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:41.247747 (+ 4798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:41.247757 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:41.247777 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:37:41.247811 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:41.249638 (+ 1827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:41.250292 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:41.250297 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:41.250311 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:41.250485 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:41.250911 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:41.250914 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:41.250915 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:41.251435 (+ 520us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:41.251546 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:41.252071 (+ 525us) server_negotiation.cc:299] Negotiation successful
0929 10:37:41.252384 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":53,"server-negotiator.queue_time_us":353,"spinlock_wait_cycles":4864,"thread_start_us":158,"threads_started":1}
I20260929 10:37:41.253233 5556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:41.236266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:38805 (local address 127.0.0.1:44694)
0929 10:37:41.236921 (+ 655us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:41.236970 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0929 10:37:41.237826 (+ 856us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:37:41.238660 (+ 834us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:37:41.238665 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:37:41.239031 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:37:41.239303 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:41.239318 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:41.241189 (+ 1871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:41.241192 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:41.241943 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:41.241952 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:41.242938 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:41.242941 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:41.243043 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:41.243712 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:37:41.243729 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:37:41.247565 (+ 3836us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:37:41.249800 (+ 2235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:37:41.249813 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:37:41.249827 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:37:41.250165 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:37:41.250583 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:37:41.250589 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:37:41.250594 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:37:41.250767 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:37:41.251582 (+ 815us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:37:41.251592 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:37:41.251797 (+ 205us) client_negotiation.cc:769] Sending connection context
0929 10:37:41.252010 (+ 213us) client_negotiation.cc:240] Negotiation successful
0929 10:37:41.252492 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438,"mutex_wait_us":105,"spinlock_wait_cycles":30848,"thread_start_us":171,"threads_started":1}
I20260929 10:37:41.255757 5553 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.82.125" port: 42085 } has no permanent_uuid. Determining permanent_uuid...
I20260929 10:37:41.256549 5556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:41.256266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.125:42085 (local address 127.0.0.1:40656)
0929 10:37:41.256407 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:41.256485 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.125:42085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260929 10:37:41.256785 5504 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.82.125:42085: connect: Connection refused (error 111)
I20260929 10:37:41.257632 5556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:41.257422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.125:42085 (local address 127.0.0.1:40672)
0929 10:37:41.257528 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:41.257574 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.125:42085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260929 10:37:41.257789 5553 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.82.125:42085: Network error: Client connection negotiation failed: client connection to 127.15.82.125:42085: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.125.keytab.
Entry for principal kudu/127.15.82.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.125.keytab.
I20260929 10:37:41.265869 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790677981592896-15689-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790677981592896-15689-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790677981592896-15689-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790677981592896-15689-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.125:42085
--webserver_interface=127.15.82.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.125.keytab
--principal=kudu/127.15.82.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:39801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.82.126:38805,127.15.82.125:42085,127.15.82.124:42611
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 10:37:41.307065 5553 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.82.125" port: 42085 } attempt: 1
I20260929 10:37:41.307950 5556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:41.307620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.125:42085 (local address 127.0.0.1:40684)
0929 10:37:41.307802 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:41.307868 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.125:42085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260929 10:37:41.308867 5556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:41.308644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.125:42085 (local address 127.0.0.1:40688)
0929 10:37:41.308757 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:41.308806 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.125:42085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260929 10:37:41.309096 5553 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.82.125:42085: Network error: Client connection negotiation failed: client connection to 127.15.82.125:42085: connect: Connection refused (error 111)
I20260929 10:37:41.377280 5553 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.82.125" port: 42085 } attempt: 2
I20260929 10:37:41.378214 5556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:41.377839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.125:42085 (local address 127.0.0.1:40698)
0929 10:37:41.378022 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:41.378101 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.125:42085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260929 10:37:41.379195 5556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:41.378965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.125:42085 (local address 127.0.0.1:40706)
0929 10:37:41.379090 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:41.379136 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.125:42085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260929 10:37:41.379352 5553 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.82.125:42085: Network error: Client connection negotiation failed: client connection to 127.15.82.125:42085: connect: Connection refused (error 111)
W20260929 10:37:41.379315 5560 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 10:37:41.379691 5560 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:37:41.379760 5560 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:37:41.379833 5560 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:37:41.383860 5560 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:37:41.383942 5560 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:37:41.383991 5560 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:37:41.384027 5560 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:37:41.384061 5560 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:37:41.389406 5560 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:39801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790677981592896-15689-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790677981592896-15689-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.15.82.126:38805,127.15.82.125:42085,127.15.82.124:42611
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.125.keytab
--principal=kudu/127.15.82.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.125:42085
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790677981592896-15689-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.15.82.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5560
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790677981592896-15689-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:37:41.390605 5560 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:37:41.391650 5560 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:37:41.400384 5566 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:41.400418 5568 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:41.400388 5565 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:37:41.400566 5560 server_base.cc:1034] running on GCE node
I20260929 10:37:41.401335 5560 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:37:41.402704 5560 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:37:41.403874 5560 hybrid_clock.cc:648] HybridClock initialized: now 1790678261403856 us; error 31 us; skew 500 ppm
Sep 29 10:37:41 dist-test-slave-vg89 krb5kdc[5472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678261, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:37:41.406970 5560 init.cc:377] Logged in from keytab as kudu/127.15.82.125@KRBTEST.COM (short username kudu)
I20260929 10:37:41.407891 5560 webserver.cc:460] Webserver started at http://127.15.82.125:45637/ using document root <none> and password file <none>
I20260929 10:37:41.408461 5560 fs_manager.cc:359] Metadata directory not provided
I20260929 10:37:41.408552 5560 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:37:41.408782 5560 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:37:41.410881 5560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790677981592896-15689-0/minicluster-data/master-1/data/instance:
uuid: "688147f3226f407da540afc3e91f4a09"
format_stamp: "Formatted at 2026-09-29 10:37:41 on dist-test-slave-vg89"
server_key: "fa8b50bb9cbc122f63a24454c2e831bc"
server_key_iv: "f347fdce5abcd3c46f5af6ee58b0d403"
server_key_version: "encryptionkey@0"
I20260929 10:37:41.411439 5560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790677981592896-15689-0/minicluster-data/master-1/wal/instance:
uuid: "688147f3226f407da540afc3e91f4a09"
format_stamp: "Formatted at 2026-09-29 10:37:41 on dist-test-slave-vg89"
server_key: "fa8b50bb9cbc122f63a24454c2e831bc"
server_key_iv: "f347fdce5abcd3c46f5af6ee58b0d403"
server_key_version: "encryptionkey@0"
I20260929 10:37:41.415148 5560 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260929 10:37:41.417596 5575 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:41.418926 5560 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 10:37:41.419067 5560 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790677981592896-15689-0/minicluster-data/master-1/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790677981592896-15689-0/minicluster-data/master-1/wal
uuid: "688147f3226f407da540afc3e91f4a09"
format_stamp: "Formatted at 2026-09-29 10:37:41 on dist-test-slave-vg89"
server_key: "fa8b50bb9cbc122f63a24454c2e831bc"
server_key_iv: "f347fdce5abcd3c46f5af6ee58b0d403"
server_key_version: "encryptionkey@0"
I20260929 10:37:41.419183 5560 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790677981592896-15689-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790677981592896-15689-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790677981592896-15689-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:37:41.432987 5560 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:37:41.436524 5560 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:37:41.436738 5560 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:37:41.443960 5560 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.125:42085
I20260929 10:37:41.444249 5627 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.125:42085 every 8 connection(s)
I20260929 10:37:41.444978 5560 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790677981592896-15689-0/minicluster-data/master-1/data/info.pb
I20260929 10:37:41.447791 5628 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:37:41.451597 5628 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.82.126" port: 38805 } has no permanent_uuid. Determining permanent_uuid...
I20260929 10:37:41.452975 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 5560
I20260929 10:37:41.453109 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790677981592896-15689-0/minicluster-data/master-1/wal/instance
I20260929 10:37:41.453449 15689 external_mini_cluster.cc:1428] Setting key d0a17a91b696380549886e7ee8c21b96
Sep 29 10:37:41 dist-test-slave-vg89 krb5kdc[5472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678261, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.125@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.82.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.124@KRBTEST.COM" created.
I20260929 10:37:41.466046 5555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:41.453556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44698 (local address 127.15.82.126:38805)
0929 10:37:41.453705 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:41.453710 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:41.454759 (+ 1049us) server_negotiation.cc:407] Connection header received
0929 10:37:41.456030 (+ 1271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:41.456033 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:41.456083 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:41.456162 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:41.457482 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:41.458028 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:41.459239 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:41.459499 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:41.459618 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:41.462036 (+ 2418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:41.462042 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:41.462049 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:37:41.462074 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:41.463526 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:41.464105 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:41.464112 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:41.464114 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:41.464197 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:41.464655 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:41.464658 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:41.464659 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:41.464912 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:41.465030 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:41.465587 (+ 557us) server_negotiation.cc:299] Negotiation successful
0929 10:37:41.465741 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260929 10:37:41.466768 5630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:41.454013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:38805 (local address 127.0.0.1:44698)
0929 10:37:41.454603 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:41.454649 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0929 10:37:41.455773 (+ 1124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:37:41.456384 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:37:41.456396 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:37:41.456850 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:37:41.457335 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:41.457348 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:41.458163 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:41.458171 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:41.459113 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:41.459122 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:41.459656 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:41.459660 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:41.459831 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:41.460451 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:37:41.460468 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:37:41.461866 (+ 1398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:37:41.463715 (+ 1849us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:37:41.463721 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:37:41.463737 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:37:41.464000 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:37:41.464349 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:37:41.464353 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:37:41.464355 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:37:41.464547 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:37:41.465051 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:37:41.465057 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:37:41.465322 (+ 265us) client_negotiation.cc:769] Sending connection context
0929 10:37:41.465492 (+ 170us) client_negotiation.cc:240] Negotiation successful
0929 10:37:41.465754 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":171,"threads_started":1}
I20260929 10:37:41.468186 5628 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.82.125" port: 42085 } has no permanent_uuid. Determining permanent_uuid...
I20260929 10:37:41.470453 5553 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.82.125" port: 42085 } attempt: 3
Sep 29 10:37:41 dist-test-slave-vg89 krb5kdc[5472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678261, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.125@KRBTEST.COM for kudu/127.15.82.125@KRBTEST.COM
Sep 29 10:37:41 dist-test-slave-vg89 krb5kdc[5472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678261, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.126@KRBTEST.COM for kudu/127.15.82.125@KRBTEST.COM
I20260929 10:37:41.480517 5556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:41.470855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.125:42085 (local address 127.0.0.1:40714)
0929 10:37:41.471016 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:41.471026 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 10:37:41.471107 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:37:41.471445 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:37:41.471448 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:37:41.471603 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:37:41.471689 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:41.471695 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:41.472898 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:41.472901 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:41.474019 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:41.474052 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:41.474633 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:41.474640 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:41.474769 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:41.475153 (+ 384us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:37:41.475164 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:37:41.476563 (+ 1399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:37:41.478915 (+ 2352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:37:41.478918 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:37:41.478919 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:37:41.479106 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:37:41.479484 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:37:41.479487 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:37:41.479488 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:37:41.479588 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:37:41.479988 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:37:41.479991 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:37:41.480093 (+ 102us) client_negotiation.cc:769] Sending connection context
0929 10:37:41.480174 (+ 81us) client_negotiation.cc:240] Negotiation successful
0929 10:37:41.480334 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102}
I20260929 10:37:41.480558 5630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:41.468731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.125:42085 (local address 127.0.0.1:40710)
0929 10:37:41.468943 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:41.468960 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 10:37:41.469044 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:37:41.469809 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:37:41.469811 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:37:41.469943 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:37:41.470069 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:41.470076 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:41.471886 (+ 1810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:41.471891 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:41.472727 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:41.472753 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:41.473581 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:41.473585 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:41.473695 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:41.474163 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:37:41.474180 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:37:41.476075 (+ 1895us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:37:41.478849 (+ 2774us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:37:41.478852 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:37:41.478856 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:37:41.479088 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:37:41.479404 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:37:41.479410 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:37:41.479415 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:37:41.479506 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:37:41.480031 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:37:41.480035 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:37:41.480118 (+ 83us) client_negotiation.cc:769] Sending connection context
0929 10:37:41.480238 (+ 120us) client_negotiation.cc:240] Negotiation successful
0929 10:37:41.480374 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"mutex_wait_us":22}
I20260929 10:37:41.480577 5634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:41.470951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40714 (local address 127.15.82.125:42085)
0929 10:37:41.471259 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:41.471266 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:41.471276 (+ 10us) server_negotiation.cc:407] Connection header received
0929 10:37:41.471306 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:41.471307 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:41.471338 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:41.471428 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:41.471809 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:41.472728 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:41.474210 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:41.474513 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:41.474590 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:41.476755 (+ 2165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:41.476758 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:41.476760 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:37:41.476779 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:41.478758 (+ 1979us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:41.479226 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:41.479228 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:41.479229 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:41.479362 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:41.479694 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:41.479697 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:41.479698 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:41.479882 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:41.480016 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:41.480297 (+ 281us) server_negotiation.cc:299] Negotiation successful
0929 10:37:41.480433 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":85,"server-negotiator.queue_time_us":247,"spinlock_wait_cycles":1280,"thread_start_us":110,"threads_started":1}
I20260929 10:37:41.481081 5633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:41.468849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40710 (local address 127.15.82.125:42085)
0929 10:37:41.469367 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:41.469379 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:41.469400 (+ 21us) server_negotiation.cc:407] Connection header received
0929 10:37:41.469441 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:41.469447 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:41.469654 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:41.469824 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:41.470201 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:41.471756 (+ 1555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:41.472873 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:41.473443 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:41.473572 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:41.476214 (+ 2642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:41.476230 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:41.476241 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:37:41.476260 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:41.478733 (+ 2473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:41.479205 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:41.479211 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:41.479215 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:41.479323 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:41.479606 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:41.479612 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:41.479617 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:41.479893 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:41.480015 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:41.480325 (+ 310us) server_negotiation.cc:299] Negotiation successful
0929 10:37:41.480518 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":67,"server-negotiator.queue_time_us":412,"spinlock_wait_cycles":2944,"thread_start_us":148,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.124.keytab.
Entry for principal kudu/127.15.82.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.124.keytab.
I20260929 10:37:41.482755 5628 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.82.124" port: 42611 } has no permanent_uuid. Determining permanent_uuid...
I20260929 10:37:41.483067 5553 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.82.124" port: 42611 } has no permanent_uuid. Determining permanent_uuid...
I20260929 10:37:41.483089 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790677981592896-15689-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790677981592896-15689-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790677981592896-15689-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790677981592896-15689-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.124:42611
--webserver_interface=127.15.82.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.124.keytab
--principal=kudu/127.15.82.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:39801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.82.126:38805,127.15.82.125:42085,127.15.82.124:42611
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 10:37:41.483536 5630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:41.483203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.124:42611 (local address 127.0.0.1:55092)
0929 10:37:41.483378 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:41.483458 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.124:42611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260929 10:37:41.483737 5556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:41.483511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.124:42611 (local address 127.0.0.1:55106)
0929 10:37:41.483626 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:41.483674 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.124:42611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260929 10:37:41.483740 5577 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.82.124:42611: connect: Connection refused (error 111)
I20260929 10:37:41.484488 5556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:41.484204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.124:42611 (local address 127.0.0.1:55116)
0929 10:37:41.484368 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:41.484408 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.124:42611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
I20260929 10:37:41.484623 5630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:41.484395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.124:42611 (local address 127.0.0.1:55120)
0929 10:37:41.484514 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:41.484563 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.124:42611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260929 10:37:41.484640 5553 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.82.124:42611: Network error: Client connection negotiation failed: client connection to 127.15.82.124:42611: connect: Connection refused (error 111)
W20260929 10:37:41.484819 5628 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.82.124:42611: Network error: Client connection negotiation failed: client connection to 127.15.82.124:42611: connect: Connection refused (error 111)
I20260929 10:37:41.515801 5553 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.82.124" port: 42611 } attempt: 1
I20260929 10:37:41.516700 5556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:41.516372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.124:42611 (local address 127.0.0.1:55124)
0929 10:37:41.516542 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:41.516612 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.124:42611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260929 10:37:41.517540 5556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:41.517314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.124:42611 (local address 127.0.0.1:55128)
0929 10:37:41.517429 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:41.517481 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.124:42611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260929 10:37:41.517696 5553 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.82.124:42611: Network error: Client connection negotiation failed: client connection to 127.15.82.124:42611: connect: Connection refused (error 111)
I20260929 10:37:41.534111 5628 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.82.124" port: 42611 } attempt: 1
I20260929 10:37:41.535128 5630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:41.534764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.124:42611 (local address 127.0.0.1:55136)
0929 10:37:41.534954 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:41.535031 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.124:42611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260929 10:37:41.535931 5630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:41.535681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.124:42611 (local address 127.0.0.1:55146)
0929 10:37:41.535809 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:41.535869 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.124:42611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260929 10:37:41.536115 5628 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.82.124:42611: Network error: Client connection negotiation failed: client connection to 127.15.82.124:42611: connect: Connection refused (error 111)
I20260929 10:37:41.592864 5553 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.82.124" port: 42611 } attempt: 2
I20260929 10:37:41.593698 5556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:41.593393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.124:42611 (local address 127.0.0.1:55148)
0929 10:37:41.593544 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:41.593616 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.124:42611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260929 10:37:41.594573 5556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:41.594317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.124:42611 (local address 127.0.0.1:55164)
0929 10:37:41.594478 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:41.594523 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.124:42611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260929 10:37:41.594700 5553 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.82.124:42611: Network error: Client connection negotiation failed: client connection to 127.15.82.124:42611: connect: Connection refused (error 111)
I20260929 10:37:41.604310 5628 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.82.124" port: 42611 } attempt: 2
I20260929 10:37:41.605084 5630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:41.604787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.124:42611 (local address 127.0.0.1:55166)
0929 10:37:41.604939 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:41.605004 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.124:42611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260929 10:37:41.605862 5630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:41.605644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.124:42611 (local address 127.0.0.1:55182)
0929 10:37:41.605749 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:41.605799 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.124:42611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260929 10:37:41.605993 5628 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.82.124:42611: Network error: Client connection negotiation failed: client connection to 127.15.82.124:42611: connect: Connection refused (error 111)
W20260929 10:37:41.606989 5636 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 10:37:41.607239 5636 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:37:41.607302 5636 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:37:41.607388 5636 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:37:41.611506 5636 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:37:41.611557 5636 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:37:41.611577 5636 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:37:41.611617 5636 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:37:41.611649 5636 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:37:41.616950 5636 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:39801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790677981592896-15689-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790677981592896-15689-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.15.82.126:38805,127.15.82.125:42085,127.15.82.124:42611
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.124.keytab
--principal=kudu/127.15.82.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.124:42611
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790677981592896-15689-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.15.82.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5636
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790677981592896-15689-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:37:41.618037 5636 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:37:41.619194 5636 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:37:41.627997 5641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:37:41.628096 5636 server_base.cc:1034] running on GCE node
W20260929 10:37:41.628001 5642 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:41.628010 5644 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:37:41.628873 5636 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:37:41.630048 5636 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:37:41.631239 5636 hybrid_clock.cc:648] HybridClock initialized: now 1790678261631205 us; error 46 us; skew 500 ppm
Sep 29 10:37:41 dist-test-slave-vg89 krb5kdc[5472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678261, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:37:41.633963 5636 init.cc:377] Logged in from keytab as kudu/127.15.82.124@KRBTEST.COM (short username kudu)
I20260929 10:37:41.634967 5636 webserver.cc:460] Webserver started at http://127.15.82.124:41435/ using document root <none> and password file <none>
I20260929 10:37:41.635530 5636 fs_manager.cc:359] Metadata directory not provided
I20260929 10:37:41.635620 5636 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:37:41.635862 5636 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:37:41.637935 5636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790677981592896-15689-0/minicluster-data/master-2/data/instance:
uuid: "e3ea36bd2db64eeb803c460a8ebd4a69"
format_stamp: "Formatted at 2026-09-29 10:37:41 on dist-test-slave-vg89"
server_key: "2c6c67f09aa7f533f5ce4778311cbfdb"
server_key_iv: "32aabdb3d131e272df830b2ec01c33fb"
server_key_version: "encryptionkey@0"
I20260929 10:37:41.638583 5636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790677981592896-15689-0/minicluster-data/master-2/wal/instance:
uuid: "e3ea36bd2db64eeb803c460a8ebd4a69"
format_stamp: "Formatted at 2026-09-29 10:37:41 on dist-test-slave-vg89"
server_key: "2c6c67f09aa7f533f5ce4778311cbfdb"
server_key_iv: "32aabdb3d131e272df830b2ec01c33fb"
server_key_version: "encryptionkey@0"
I20260929 10:37:41.642249 5636 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260929 10:37:41.644708 5651 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:41.645934 5636 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 10:37:41.646181 5636 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790677981592896-15689-0/minicluster-data/master-2/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790677981592896-15689-0/minicluster-data/master-2/wal
uuid: "e3ea36bd2db64eeb803c460a8ebd4a69"
format_stamp: "Formatted at 2026-09-29 10:37:41 on dist-test-slave-vg89"
server_key: "2c6c67f09aa7f533f5ce4778311cbfdb"
server_key_iv: "32aabdb3d131e272df830b2ec01c33fb"
server_key_version: "encryptionkey@0"
I20260929 10:37:41.646307 5636 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790677981592896-15689-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790677981592896-15689-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790677981592896-15689-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:37:41.684767 5636 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:37:41.688138 5636 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:37:41.688365 5636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:37:41.693895 5553 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.82.124" port: 42611 } attempt: 3
I20260929 10:37:41.694836 5556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:41.694447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.124:42611 (local address 127.0.0.1:55184)
0929 10:37:41.694602 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:41.694705 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.124:42611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260929 10:37:41.695546 5556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:41.695319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.124:42611 (local address 127.0.0.1:55198)
0929 10:37:41.695430 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:41.695479 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.124:42611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260929 10:37:41.695737 5553 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.82.124:42611: Network error: Client connection negotiation failed: client connection to 127.15.82.124:42611: connect: Connection refused (error 111)
I20260929 10:37:41.696007 5636 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.124:42611
I20260929 10:37:41.696290 5703 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.124:42611 every 8 connection(s)
I20260929 10:37:41.697177 5636 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790677981592896-15689-0/minicluster-data/master-2/data/info.pb
I20260929 10:37:41.697145 5628 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.82.124" port: 42611 } attempt: 3
I20260929 10:37:41.700433 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 5636
I20260929 10:37:41.700515 5704 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:37:41.700567 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790677981592896-15689-0/minicluster-data/master-2/wal/instance
I20260929 10:37:41.700876 15689 external_mini_cluster.cc:1428] Setting key 06464ddab08ddf19dfe46d521b3695f1
I20260929 10:37:41.705618 5704 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.82.126" port: 38805 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 10:37:41 dist-test-slave-vg89 krb5kdc[5472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678261, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.125@KRBTEST.COM for kudu/127.15.82.124@KRBTEST.COM
Sep 29 10:37:41 dist-test-slave-vg89 krb5kdc[5472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678260, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.82.124@KRBTEST.COM
I20260929 10:37:41.711194 5630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:41.697586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.124:42611 (local address 127.0.0.1:55210)
0929 10:37:41.697726 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:41.697745 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 10:37:41.697844 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:37:41.700356 (+ 2512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:37:41.700360 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:37:41.700576 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:37:41.700693 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:41.700701 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:41.702665 (+ 1964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:41.702670 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:41.703197 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:41.703206 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:41.704184 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:41.704187 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:41.704285 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:41.704894 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:37:41.704910 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:37:41.706588 (+ 1678us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:37:41.709146 (+ 2558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:37:41.709150 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:37:41.709152 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:37:41.709463 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:37:41.709863 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:37:41.709868 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:37:41.709870 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:37:41.709963 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:37:41.710681 (+ 718us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:37:41.710684 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:37:41.710787 (+ 103us) client_negotiation.cc:769] Sending connection context
0929 10:37:41.710917 (+ 130us) client_negotiation.cc:240] Negotiation successful
0929 10:37:41.711066 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260929 10:37:41.712499 5706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:41.698388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55210 (local address 127.15.82.124:42611)
0929 10:37:41.698925 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:41.698941 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:41.698978 (+ 37us) server_negotiation.cc:407] Connection header received
0929 10:37:41.699676 (+ 698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:41.699695 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:41.700044 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:41.700423 (+ 379us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:41.701004 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:41.702540 (+ 1536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:41.703341 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:41.704037 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:41.704591 (+ 554us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:41.706800 (+ 2209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:41.706807 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:41.706823 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:37:41.706852 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:41.709010 (+ 2158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:41.709607 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:41.709612 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:41.709618 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:41.709753 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:41.710078 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:41.710081 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:41.710083 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:41.710567 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:41.710695 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:41.711129 (+ 434us) server_negotiation.cc:299] Negotiation successful
0929 10:37:41.711431 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":139,"threads_started":1}
I20260929 10:37:41.713027 5708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:41.701886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55224 (local address 127.15.82.124:42611)
0929 10:37:41.702158 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:41.702166 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:41.702179 (+ 13us) server_negotiation.cc:407] Connection header received
0929 10:37:41.702212 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:41.702214 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:41.702260 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:41.702344 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:41.703284 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:41.703855 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:41.704772 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:41.705050 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:41.705165 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:41.708006 (+ 2841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:41.708010 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:41.708012 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:37:41.708035 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:41.709758 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:41.710501 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:41.710504 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:41.710506 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:41.710582 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:41.711758 (+ 1176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:41.711761 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:41.711763 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:41.712012 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:41.712199 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:41.712421 (+ 222us) server_negotiation.cc:299] Negotiation successful
0929 10:37:41.712545 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":216,"thread_start_us":107,"threads_started":1}
Sep 29 10:37:41 dist-test-slave-vg89 krb5kdc[5472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678261, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.124@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:37:41.717481 5628 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 688147f3226f407da540afc3e91f4a09: Bootstrap starting.
I20260929 10:37:41.719218 5555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:41.707778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44712 (local address 127.15.82.126:38805)
0929 10:37:41.707949 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:41.707954 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:41.708852 (+ 898us) server_negotiation.cc:407] Connection header received
0929 10:37:41.709058 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:41.709064 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:41.709140 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:41.709227 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:41.710068 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:41.710859 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:41.712046 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:41.712396 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:41.712497 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:41.715729 (+ 3232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:41.715732 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:41.715737 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:37:41.715766 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:41.717264 (+ 1498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:41.717834 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:41.717837 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:41.717839 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:41.717921 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:41.718204 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:41.718207 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:41.718208 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:41.718520 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:41.718626 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:41.718949 (+ 323us) server_negotiation.cc:299] Negotiation successful
0929 10:37:41.719061 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":71}
I20260929 10:37:41.719233 5709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:41.707709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:38805 (local address 127.0.0.1:44712)
0929 10:37:41.708677 (+ 968us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:41.708735 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0929 10:37:41.708876 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:37:41.709282 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:37:41.709289 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:37:41.709794 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:37:41.709948 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:41.709954 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:41.710986 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:41.710989 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:41.711907 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:41.711917 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:41.712629 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:41.712634 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:41.712741 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:41.713956 (+ 1215us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:37:41.713970 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:37:41.715564 (+ 1594us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:37:41.717398 (+ 1834us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:37:41.717405 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:37:41.717409 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:37:41.717689 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:37:41.718014 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:37:41.718019 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:37:41.718020 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:37:41.718105 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:37:41.718623 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:37:41.718629 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:37:41.718747 (+ 118us) client_negotiation.cc:769] Sending connection context
0929 10:37:41.718885 (+ 138us) client_negotiation.cc:240] Negotiation successful
0929 10:37:41.719061 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":787,"thread_start_us":111,"threads_started":1}
I20260929 10:37:41.720247 5704 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.82.125" port: 42085 } has no permanent_uuid. Determining permanent_uuid...
I20260929 10:37:41.720455 5628 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 688147f3226f407da540afc3e91f4a09: Neither blocks nor log segments found. Creating new log.
I20260929 10:37:41.721387 5628 log.cc:824] T 00000000000000000000000000000000 P 688147f3226f407da540afc3e91f4a09: Log is configured to *not* fsync() on all Append() calls
I20260929 10:37:41.723488 5628 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 688147f3226f407da540afc3e91f4a09: No bootstrap required, opened a new log
Sep 29 10:37:41 dist-test-slave-vg89 krb5kdc[5472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678260, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
Sep 29 10:37:41 dist-test-slave-vg89 krb5kdc[5472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678261, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.124@KRBTEST.COM for kudu/127.15.82.125@KRBTEST.COM
I20260929 10:37:41.728183 5628 raft_consensus.cc:348] T 00000000000000000000000000000000 P 688147f3226f407da540afc3e91f4a09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5173f7f7114d495a8774942645187c35" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 38805 } } peers { permanent_uuid: "688147f3226f407da540afc3e91f4a09" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 42085 } } peers { permanent_uuid: "e3ea36bd2db64eeb803c460a8ebd4a69" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 42611 } }
I20260929 10:37:41.728447 5628 raft_consensus.cc:729] T 00000000000000000000000000000000 P 688147f3226f407da540afc3e91f4a09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 688147f3226f407da540afc3e91f4a09, State: Initialized, Role: FOLLOWER
I20260929 10:37:41.728863 5710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:41.715496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44724 (local address 127.15.82.126:38805)
0929 10:37:41.716548 (+ 1052us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:41.716560 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:41.716574 (+ 14us) server_negotiation.cc:407] Connection header received
0929 10:37:41.716630 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:41.717373 (+ 743us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:41.717422 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:41.717497 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:41.718380 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:41.719235 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:41.720838 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:41.721166 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:41.721284 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:41.724841 (+ 3557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:41.724846 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:41.724848 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:37:41.724875 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:41.726685 (+ 1810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:41.727141 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:41.727147 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:41.727148 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:41.727248 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:41.727610 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:41.727615 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:41.727618 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:41.727870 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:41.728111 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:41.728584 (+ 473us) server_negotiation.cc:299] Negotiation successful
0929 10:37:41.728707 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":599,"thread_start_us":90,"threads_started":1}
I20260929 10:37:41.728829 5628 consensus_queue.cc:260] T 00000000000000000000000000000000 P 688147f3226f407da540afc3e91f4a09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5173f7f7114d495a8774942645187c35" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 38805 } } peers { permanent_uuid: "688147f3226f407da540afc3e91f4a09" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 42085 } } peers { permanent_uuid: "e3ea36bd2db64eeb803c460a8ebd4a69" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 42611 } }
I20260929 10:37:41.730659 5628 sys_catalog.cc:564] T 00000000000000000000000000000000 P 688147f3226f407da540afc3e91f4a09 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:37:41.730669 5712 sys_catalog.cc:455] T 00000000000000000000000000000000 P 688147f3226f407da540afc3e91f4a09 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5173f7f7114d495a8774942645187c35" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 38805 } } peers { permanent_uuid: "688147f3226f407da540afc3e91f4a09" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 42085 } } peers { permanent_uuid: "e3ea36bd2db64eeb803c460a8ebd4a69" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 42611 } } }
I20260929 10:37:41.730839 5709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:41.720757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.125:42085 (local address 127.0.0.1:40720)
0929 10:37:41.721017 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:41.721028 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:37:41.721123 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:37:41.721435 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:37:41.721438 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:37:41.721636 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:37:41.721758 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:41.721777 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:41.722577 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:41.722582 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:41.723533 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:41.723546 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:41.724158 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:41.724163 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:41.724272 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:41.724722 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:37:41.724741 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:37:41.727052 (+ 2311us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:37:41.729102 (+ 2050us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:37:41.729108 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:37:41.729113 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:37:41.729359 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:37:41.729673 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:37:41.729680 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:37:41.729684 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:37:41.729784 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:37:41.730293 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:37:41.730296 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:37:41.730389 (+ 93us) client_negotiation.cc:769] Sending connection context
0929 10:37:41.730534 (+ 145us) client_negotiation.cc:240] Negotiation successful
0929 10:37:41.730677 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208}
I20260929 10:37:41.730845 5712 sys_catalog.cc:458] T 00000000000000000000000000000000 P 688147f3226f407da540afc3e91f4a09 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 10:37:41.731307 5633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:41.720816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40720 (local address 127.15.82.125:42085)
0929 10:37:41.720915 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:41.720920 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:41.721089 (+ 169us) server_negotiation.cc:407] Connection header received
0929 10:37:41.721206 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:41.721209 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:41.721261 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:41.721330 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:41.721884 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:41.722463 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:41.723687 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:41.724042 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:41.724153 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:41.727221 (+ 3068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:41.727225 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:41.727228 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:37:41.727254 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:41.728971 (+ 1717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:41.729469 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:41.729473 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:41.729474 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:41.729568 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:41.729883 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:41.729886 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:41.729888 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:41.730137 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:41.730235 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:41.731026 (+ 791us) server_negotiation.cc:299] Negotiation successful
0929 10:37:41.731159 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260929 10:37:41.732079 5704 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.82.124" port: 42611 } has no permanent_uuid. Determining permanent_uuid...
W20260929 10:37:41.736526 5723 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 688147f3226f407da540afc3e91f4a09: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 10:37:41.736599 5723 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 10:37:41.734987 5560 master_runner.cc:186] Error getting master registration for 127.15.82.126:38805: OK, instance_id { permanent_uuid: "5173f7f7114d495a8774942645187c35" instance_seqno: 1790678261223856 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 10:37:41.738303 5560 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Sep 29 10:37:41 dist-test-slave-vg89 krb5kdc[5472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678260, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.82.125@KRBTEST.COM
Sep 29 10:37:41 dist-test-slave-vg89 krb5kdc[5472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678261, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.124@KRBTEST.COM for kudu/127.15.82.124@KRBTEST.COM
I20260929 10:37:41.743070 5709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:41.732546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.124:42611 (local address 127.0.0.1:55238)
0929 10:37:41.732777 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:41.732793 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 10:37:41.732913 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:37:41.733691 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:37:41.733697 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:37:41.733889 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:37:41.733976 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:41.733982 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:41.735062 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:41.735065 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:41.735913 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:41.735931 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:41.736791 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:41.736795 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:41.736927 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:41.737455 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:37:41.737471 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:37:41.739439 (+ 1968us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:37:41.741358 (+ 1919us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:37:41.741362 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:37:41.741364 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:37:41.741620 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:37:41.741909 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:37:41.741912 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:37:41.741914 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:37:41.741990 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:37:41.742536 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:37:41.742539 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:37:41.742648 (+ 109us) client_negotiation.cc:769] Sending connection context
0929 10:37:41.742753 (+ 105us) client_negotiation.cc:240] Negotiation successful
0929 10:37:41.742896 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151}
I20260929 10:37:41.743364 5634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:41.730896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40736 (local address 127.15.82.125:42085)
0929 10:37:41.731135 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:41.731138 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:41.731497 (+ 359us) server_negotiation.cc:407] Connection header received
0929 10:37:41.731669 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:41.731671 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:41.731738 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:41.731829 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:41.732662 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:41.733604 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:41.734836 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:41.735233 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:41.735347 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:41.739063 (+ 3716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:41.739067 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:41.739070 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:37:41.739095 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:41.741151 (+ 2056us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:41.741667 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:41.741671 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:41.741674 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:41.741766 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:41.742090 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:41.742093 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:41.742095 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:41.742296 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:41.742552 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:41.742737 (+ 185us) server_negotiation.cc:299] Negotiation successful
0929 10:37:41.742876 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171}
I20260929 10:37:41.743511 5708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:41.733048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55238 (local address 127.15.82.124:42611)
0929 10:37:41.733423 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:41.733428 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:41.733441 (+ 13us) server_negotiation.cc:407] Connection header received
0929 10:37:41.733475 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:41.733477 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:41.733515 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:41.733587 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:41.734097 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:41.734956 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:41.736059 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:41.736456 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:41.736582 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:41.739584 (+ 3002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:41.739590 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:41.739595 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:37:41.739620 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:41.741258 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:41.741729 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:41.741732 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:41.741734 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:41.741805 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:41.742075 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:41.742078 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:41.742079 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:41.742349 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:41.742508 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:41.743270 (+ 762us) server_negotiation.cc:299] Negotiation successful
0929 10:37:41.743379 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305}
I20260929 10:37:41.747071 5704 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3ea36bd2db64eeb803c460a8ebd4a69: Bootstrap starting.
I20260929 10:37:41.747191 15689 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 10:37:41.750190 5704 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e3ea36bd2db64eeb803c460a8ebd4a69: Neither blocks nor log segments found. Creating new log.
I20260929 10:37:41.751144 5704 log.cc:824] T 00000000000000000000000000000000 P e3ea36bd2db64eeb803c460a8ebd4a69: Log is configured to *not* fsync() on all Append() calls
I20260929 10:37:41.749836 5527 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "5173f7f7114d495a8774942645187c35"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44724
I20260929 10:37:41.753230 5704 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3ea36bd2db64eeb803c460a8ebd4a69: No bootstrap required, opened a new log
I20260929 10:37:41.757948 5704 raft_consensus.cc:348] T 00000000000000000000000000000000 P e3ea36bd2db64eeb803c460a8ebd4a69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5173f7f7114d495a8774942645187c35" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 38805 } } peers { permanent_uuid: "688147f3226f407da540afc3e91f4a09" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 42085 } } peers { permanent_uuid: "e3ea36bd2db64eeb803c460a8ebd4a69" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 42611 } }
I20260929 10:37:41.758258 5704 raft_consensus.cc:729] T 00000000000000000000000000000000 P e3ea36bd2db64eeb803c460a8ebd4a69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3ea36bd2db64eeb803c460a8ebd4a69, State: Initialized, Role: FOLLOWER
I20260929 10:37:41.758694 5704 consensus_queue.cc:260] T 00000000000000000000000000000000 P e3ea36bd2db64eeb803c460a8ebd4a69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5173f7f7114d495a8774942645187c35" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 38805 } } peers { permanent_uuid: "688147f3226f407da540afc3e91f4a09" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 42085 } } peers { permanent_uuid: "e3ea36bd2db64eeb803c460a8ebd4a69" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 42611 } }
I20260929 10:37:41.759356 5726 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3ea36bd2db64eeb803c460a8ebd4a69 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5173f7f7114d495a8774942645187c35" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 38805 } } peers { permanent_uuid: "688147f3226f407da540afc3e91f4a09" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 42085 } } peers { permanent_uuid: "e3ea36bd2db64eeb803c460a8ebd4a69" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 42611 } } }
I20260929 10:37:41.759652 5726 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3ea36bd2db64eeb803c460a8ebd4a69 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 10:37:41.760304 5704 sys_catalog.cc:564] T 00000000000000000000000000000000 P e3ea36bd2db64eeb803c460a8ebd4a69 [sys.catalog]: configured and running, proceeding with master startup.
W20260929 10:37:41.766754 5737 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e3ea36bd2db64eeb803c460a8ebd4a69: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 10:37:41.766930 5737 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 10:37:41.764919 5636 master_runner.cc:186] Error getting master registration for 127.15.82.126:38805: OK, instance_id { permanent_uuid: "5173f7f7114d495a8774942645187c35" instance_seqno: 1790678261223856 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 10:37:41.768604 5636 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 10:37:41.859895 5553 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.82.124" port: 42611 } attempt: 4
Sep 29 10:37:41 dist-test-slave-vg89 krb5kdc[5472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678261, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.126@KRBTEST.COM for kudu/127.15.82.124@KRBTEST.COM
I20260929 10:37:41.870661 5708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:41.861026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55252 (local address 127.15.82.124:42611)
0929 10:37:41.861158 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:41.861163 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:41.861177 (+ 14us) server_negotiation.cc:407] Connection header received
0929 10:37:41.861225 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:41.861228 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:41.861279 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:41.861366 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:41.862074 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:41.862867 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:41.864083 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:41.864470 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:41.864573 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:41.867035 (+ 2462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:41.867039 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:41.867042 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:37:41.867071 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:41.868704 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:41.869230 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:41.869234 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:41.869236 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:41.869330 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:41.869654 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:41.869658 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:41.869660 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:41.869960 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:41.870063 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:41.870343 (+ 280us) server_negotiation.cc:299] Negotiation successful
0929 10:37:41.870484 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260929 10:37:41.870661 5556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:41.860652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.124:42611 (local address 127.0.0.1:55252)
0929 10:37:41.860823 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:41.860844 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 10:37:41.860960 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:37:41.861518 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:37:41.861521 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:37:41.861819 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:37:41.861946 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:41.861954 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:41.862995 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:41.863000 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:41.863938 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:41.863948 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:41.864580 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:41.864584 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:41.864677 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:41.865238 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:37:41.865253 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:37:41.866892 (+ 1639us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:37:41.868855 (+ 1963us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:37:41.868859 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:37:41.868862 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:37:41.869139 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:37:41.869435 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:37:41.869440 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:37:41.869444 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:37:41.869542 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:37:41.870057 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:37:41.870060 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:37:41.870171 (+ 111us) client_negotiation.cc:769] Sending connection context
0929 10:37:41.870289 (+ 118us) client_negotiation.cc:240] Negotiation successful
0929 10:37:41.870476 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260929 10:37:41.874555 5553 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5173f7f7114d495a8774942645187c35: Bootstrap starting.
I20260929 10:37:41.877616 5553 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5173f7f7114d495a8774942645187c35: Neither blocks nor log segments found. Creating new log.
I20260929 10:37:41.878582 5553 log.cc:824] T 00000000000000000000000000000000 P 5173f7f7114d495a8774942645187c35: Log is configured to *not* fsync() on all Append() calls
I20260929 10:37:41.880928 5553 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5173f7f7114d495a8774942645187c35: No bootstrap required, opened a new log
I20260929 10:37:41.884291 5553 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5173f7f7114d495a8774942645187c35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5173f7f7114d495a8774942645187c35" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 38805 } } peers { permanent_uuid: "688147f3226f407da540afc3e91f4a09" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 42085 } } peers { permanent_uuid: "e3ea36bd2db64eeb803c460a8ebd4a69" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 42611 } }
I20260929 10:37:41.884536 5553 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5173f7f7114d495a8774942645187c35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5173f7f7114d495a8774942645187c35, State: Initialized, Role: FOLLOWER
I20260929 10:37:41.884910 5553 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5173f7f7114d495a8774942645187c35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5173f7f7114d495a8774942645187c35" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 38805 } } peers { permanent_uuid: "688147f3226f407da540afc3e91f4a09" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 42085 } } peers { permanent_uuid: "e3ea36bd2db64eeb803c460a8ebd4a69" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 42611 } }
I20260929 10:37:41.885505 5740 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5173f7f7114d495a8774942645187c35 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5173f7f7114d495a8774942645187c35" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 38805 } } peers { permanent_uuid: "688147f3226f407da540afc3e91f4a09" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 42085 } } peers { permanent_uuid: "e3ea36bd2db64eeb803c460a8ebd4a69" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 42611 } } }
I20260929 10:37:41.885728 5740 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5173f7f7114d495a8774942645187c35 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 10:37:41.886375 5553 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5173f7f7114d495a8774942645187c35 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:37:41.886432 5527 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5173f7f7114d495a8774942645187c35 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260929 10:37:41.886530 5527 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5173f7f7114d495a8774942645187c35 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:37:41.888690 5527 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5173f7f7114d495a8774942645187c35 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5173f7f7114d495a8774942645187c35" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 38805 } } peers { permanent_uuid: "688147f3226f407da540afc3e91f4a09" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 42085 } } peers { permanent_uuid: "e3ea36bd2db64eeb803c460a8ebd4a69" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 42611 } }
I20260929 10:37:41.890045 5527 leader_election.cc:288] T 00000000000000000000000000000000 P 5173f7f7114d495a8774942645187c35 [CANDIDATE]: Term 1 election: Requested vote from peers 688147f3226f407da540afc3e91f4a09 (127.15.82.125:42085), e3ea36bd2db64eeb803c460a8ebd4a69 (127.15.82.124:42611)
I20260929 10:37:41.890554 5601 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5173f7f7114d495a8774942645187c35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "688147f3226f407da540afc3e91f4a09"
I20260929 10:37:41.890677 5678 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5173f7f7114d495a8774942645187c35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "e3ea36bd2db64eeb803c460a8ebd4a69"
I20260929 10:37:41.890830 5601 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 688147f3226f407da540afc3e91f4a09 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:37:41.890880 5678 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e3ea36bd2db64eeb803c460a8ebd4a69 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:37:41.892194 5678 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e3ea36bd2db64eeb803c460a8ebd4a69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5173f7f7114d495a8774942645187c35 in term 1.
I20260929 10:37:41.892194 5601 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 688147f3226f407da540afc3e91f4a09 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5173f7f7114d495a8774942645187c35 in term 1.
I20260929 10:37:41.892693 5502 leader_election.cc:302] T 00000000000000000000000000000000 P 5173f7f7114d495a8774942645187c35 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5173f7f7114d495a8774942645187c35, e3ea36bd2db64eeb803c460a8ebd4a69; no voters:
I20260929 10:37:41.894596 5740 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5173f7f7114d495a8774942645187c35 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:37:41.894873 5740 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5173f7f7114d495a8774942645187c35 [term 1 LEADER]: Becoming Leader. State: Replica: 5173f7f7114d495a8774942645187c35, State: Running, Role: LEADER
I20260929 10:37:41.895272 5740 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5173f7f7114d495a8774942645187c35 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5173f7f7114d495a8774942645187c35" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 38805 } } peers { permanent_uuid: "688147f3226f407da540afc3e91f4a09" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 42085 } } peers { permanent_uuid: "e3ea36bd2db64eeb803c460a8ebd4a69" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 42611 } }
I20260929 10:37:41.898553 5740 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5173f7f7114d495a8774942645187c35 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5173f7f7114d495a8774942645187c35. Latest consensus state: current_term: 1 leader_uuid: "5173f7f7114d495a8774942645187c35" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5173f7f7114d495a8774942645187c35" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 38805 } } peers { permanent_uuid: "688147f3226f407da540afc3e91f4a09" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 42085 } } peers { permanent_uuid: "e3ea36bd2db64eeb803c460a8ebd4a69" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 42611 } } }
I20260929 10:37:41.898693 5740 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5173f7f7114d495a8774942645187c35 [sys.catalog]: This master's current role is: LEADER
W20260929 10:37:41.899330 5751 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5173f7f7114d495a8774942645187c35: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 10:37:41.899591 5751 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 10:37:41.899605 5485 master_runner.cc:263] No leader master found from master 5173f7f7114d495a8774942645187c35
I20260929 10:37:41.899674 5485 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 10:37:41.907467 5634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:41.895285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40740 (local address 127.15.82.125:42085)
0929 10:37:41.895918 (+ 633us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:41.895922 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:41.897572 (+ 1650us) server_negotiation.cc:407] Connection header received
0929 10:37:41.897766 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:41.897769 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:41.897821 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:41.897876 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:41.899113 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:41.899878 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:41.900852 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:41.901186 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:41.901296 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:41.903444 (+ 2148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:41.903451 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:41.903457 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:37:41.903488 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:41.905471 (+ 1983us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:41.905993 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:41.905997 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:41.905999 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:41.906086 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:41.906379 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:41.906383 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:41.906384 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:41.906659 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:41.906781 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:41.907162 (+ 381us) server_negotiation.cc:299] Negotiation successful
0929 10:37:41.907286 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":551}
I20260929 10:37:41.922116 5762 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:37:41.923537 5762 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:37:41.924324 5708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:41.896576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55260 (local address 127.15.82.124:42611)
0929 10:37:41.896688 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:41.896692 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:41.902738 (+ 6046us) server_negotiation.cc:407] Connection header received
0929 10:37:41.903210 (+ 472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:41.903214 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:41.903269 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:41.903346 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:41.905550 (+ 2204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:41.906425 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:41.911158 (+ 4733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:41.911537 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:41.911648 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:41.918831 (+ 7183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:41.918839 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:41.918843 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:37:41.918876 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:41.920714 (+ 1838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:41.921670 (+ 956us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:41.921673 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:41.921675 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:41.921759 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:41.922328 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:41.922332 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:41.922334 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:41.922619 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:41.922755 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:41.924002 (+ 1247us) server_negotiation.cc:299] Negotiation successful
0929 10:37:41.924147 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260929 10:37:41.926589 5710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:41.896168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44740 (local address 127.15.82.126:38805)
0929 10:37:41.896275 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:41.896281 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:41.903001 (+ 6720us) server_negotiation.cc:407] Connection header received
0929 10:37:41.904463 (+ 1462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:41.904468 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:41.904525 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:41.904595 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:41.907711 (+ 3116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:41.908449 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:41.910603 (+ 2154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:41.910931 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:41.910999 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:41.917644 (+ 6645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:41.917649 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:41.917652 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:37:41.917682 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:41.919778 (+ 2096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:41.921633 (+ 1855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:41.921641 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:41.921646 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:41.921750 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:41.923158 (+ 1408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:41.923162 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:41.923164 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:41.923410 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:41.923525 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:41.926261 (+ 2736us) server_negotiation.cc:299] Negotiation successful
0929 10:37:41.926435 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260929 10:37:41.930665 5678 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e3ea36bd2db64eeb803c460a8ebd4a69 [term 1 FOLLOWER]: Refusing update from remote peer 5173f7f7114d495a8774942645187c35: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:37:41.931442 5740 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5173f7f7114d495a8774942645187c35 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3ea36bd2db64eeb803c460a8ebd4a69" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 42611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:37:41.931574 5601 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 688147f3226f407da540afc3e91f4a09 [term 1 FOLLOWER]: Refusing update from remote peer 5173f7f7114d495a8774942645187c35: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:37:41.932288 5757 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5173f7f7114d495a8774942645187c35 [LEADER]: Connected to new peer: Peer: permanent_uuid: "688147f3226f407da540afc3e91f4a09" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 42085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:37:41.936309 5712 sys_catalog.cc:455] T 00000000000000000000000000000000 P 688147f3226f407da540afc3e91f4a09 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5173f7f7114d495a8774942645187c35. Latest consensus state: current_term: 1 leader_uuid: "5173f7f7114d495a8774942645187c35" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5173f7f7114d495a8774942645187c35" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 38805 } } peers { permanent_uuid: "688147f3226f407da540afc3e91f4a09" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 42085 } } peers { permanent_uuid: "e3ea36bd2db64eeb803c460a8ebd4a69" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 42611 } } }
I20260929 10:37:41.936450 5712 sys_catalog.cc:458] T 00000000000000000000000000000000 P 688147f3226f407da540afc3e91f4a09 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 10:37:41.936547 5726 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3ea36bd2db64eeb803c460a8ebd4a69 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5173f7f7114d495a8774942645187c35. Latest consensus state: current_term: 1 leader_uuid: "5173f7f7114d495a8774942645187c35" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5173f7f7114d495a8774942645187c35" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 38805 } } peers { permanent_uuid: "688147f3226f407da540afc3e91f4a09" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 42085 } } peers { permanent_uuid: "e3ea36bd2db64eeb803c460a8ebd4a69" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 42611 } } }
I20260929 10:37:41.936685 5726 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3ea36bd2db64eeb803c460a8ebd4a69 [sys.catalog]: This master's current role is: FOLLOWER
W20260929 10:37:41.938313 5581 tablet.cc:2369] T 00000000000000000000000000000000 P 688147f3226f407da540afc3e91f4a09: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 10:37:41.939805 5657 tablet.cc:2369] T 00000000000000000000000000000000 P e3ea36bd2db64eeb803c460a8ebd4a69: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 10:37:41.942265 5757 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5173f7f7114d495a8774942645187c35 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5173f7f7114d495a8774942645187c35" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5173f7f7114d495a8774942645187c35" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 38805 } } peers { permanent_uuid: "688147f3226f407da540afc3e91f4a09" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 42085 } } peers { permanent_uuid: "e3ea36bd2db64eeb803c460a8ebd4a69" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 42611 } } }
I20260929 10:37:41.942432 5757 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5173f7f7114d495a8774942645187c35 [sys.catalog]: This master's current role is: LEADER
I20260929 10:37:41.943888 5712 sys_catalog.cc:455] T 00000000000000000000000000000000 P 688147f3226f407da540afc3e91f4a09 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5173f7f7114d495a8774942645187c35" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5173f7f7114d495a8774942645187c35" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 38805 } } peers { permanent_uuid: "688147f3226f407da540afc3e91f4a09" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 42085 } } peers { permanent_uuid: "e3ea36bd2db64eeb803c460a8ebd4a69" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 42611 } } }
I20260929 10:37:41.944022 5712 sys_catalog.cc:458] T 00000000000000000000000000000000 P 688147f3226f407da540afc3e91f4a09 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 10:37:41.944160 5763 mvcc.cc:205] Tried to move back new op lower bound from 7334618160858963968 to 7334618160731947008. Current Snapshot: MvccSnapshot[applied={T|T < 7334618160858963968}]
I20260929 10:37:41.944552 5764 mvcc.cc:205] Tried to move back new op lower bound from 7334618160858963968 to 7334618160731947008. Current Snapshot: MvccSnapshot[applied={T|T < 7334618160858963968}]
I20260929 10:37:41.949064 5762 catalog_manager.cc:1347] Generated new cluster ID: 54ef35191d4e46c4a9cd436e467b6f26
I20260929 10:37:41.949146 5762 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:37:41.950021 5740 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5173f7f7114d495a8774942645187c35 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5173f7f7114d495a8774942645187c35" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5173f7f7114d495a8774942645187c35" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 38805 } } peers { permanent_uuid: "688147f3226f407da540afc3e91f4a09" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 42085 } } peers { permanent_uuid: "e3ea36bd2db64eeb803c460a8ebd4a69" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 42611 } } }
I20260929 10:37:41.950168 5740 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5173f7f7114d495a8774942645187c35 [sys.catalog]: This master's current role is: LEADER
I20260929 10:37:41.951179 5726 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3ea36bd2db64eeb803c460a8ebd4a69 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5173f7f7114d495a8774942645187c35" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5173f7f7114d495a8774942645187c35" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 38805 } } peers { permanent_uuid: "688147f3226f407da540afc3e91f4a09" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 42085 } } peers { permanent_uuid: "e3ea36bd2db64eeb803c460a8ebd4a69" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 42611 } } }
I20260929 10:37:41.951328 5726 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3ea36bd2db64eeb803c460a8ebd4a69 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 10:37:41.951934 5765 mvcc.cc:205] Tried to move back new op lower bound from 7334618160858963968 to 7334618160731947008. Current Snapshot: MvccSnapshot[applied={T|T < 7334618160858963968}]
I20260929 10:37:41.998176 5762 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:37:41.999600 5762 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:37:42.007512 5762 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5173f7f7114d495a8774942645187c35: Generated new TSK 0
I20260929 10:37:42.008384 5762 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260929 10:37:42.030445 5708 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:37:42.028852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55272 (local address 127.15.82.124:42611)
0929 10:37:42.028974 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:42.028980 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:42.029450 (+ 470us) server_negotiation.cc:407] Connection header received
0929 10:37:42.029657 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:42.029661 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:42.029715 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:42.029803 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:42.030269 (+ 466us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55272: BlockingRecv error: recv got EOF from 127.0.0.1:55272 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260929 10:37:42.030433 5710 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:37:42.028867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44748 (local address 127.15.82.126:38805)
0929 10:37:42.029105 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:42.029109 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:42.029234 (+ 125us) server_negotiation.cc:407] Connection header received
0929 10:37:42.029459 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:42.029464 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:42.029521 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:42.029581 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:42.030269 (+ 688us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44748: BlockingRecv error: recv got EOF from 127.0.0.1:44748 (error 108)
Metrics: {"server-negotiator.queue_time_us":165}
W20260929 10:37:42.031355 5634 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:37:42.028716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40742 (local address 127.15.82.125:42085)
0929 10:37:42.029009 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:42.029014 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:42.030114 (+ 1100us) server_negotiation.cc:407] Connection header received
0929 10:37:42.030342 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:42.030346 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:42.030419 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:42.030476 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:42.031206 (+ 730us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40742: BlockingRecv error: recv got EOF from 127.0.0.1:40742 (error 108)
Metrics: {"server-negotiator.queue_time_us":207}
W20260929 10:37:42.034343 5634 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:37:42.033202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40748 (local address 127.15.82.125:42085)
0929 10:37:42.033324 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:42.033328 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:42.033478 (+ 150us) server_negotiation.cc:407] Connection header received
0929 10:37:42.033668 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:42.033670 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:42.033730 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:42.033796 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:42.034247 (+ 451us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40748: BlockingRecv error: recv got EOF from 127.0.0.1:40748 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260929 10:37:42.038694 5634 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:37:42.037807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40764 (local address 127.15.82.125:42085)
0929 10:37:42.037929 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:42.037934 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:42.037953 (+ 19us) server_negotiation.cc:407] Connection header received
0929 10:37:42.037995 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:42.037997 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:42.038034 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:42.038125 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:42.038596 (+ 471us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40764: BlockingRecv error: recv got EOF from 127.0.0.1:40764 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260929 10:37:42.043535 5634 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:37:42.042455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40778 (local address 127.15.82.125:42085)
0929 10:37:42.042562 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:42.042566 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:42.042695 (+ 129us) server_negotiation.cc:407] Connection header received
0929 10:37:42.042887 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:42.042891 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:42.042938 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:42.043028 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:42.043416 (+ 388us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40778: BlockingRecv error: recv got EOF from 127.0.0.1:40778 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260929 10:37:42.050350 5634 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:37:42.049281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40786 (local address 127.15.82.125:42085)
0929 10:37:42.049429 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:42.049433 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:42.049587 (+ 154us) server_negotiation.cc:407] Connection header received
0929 10:37:42.049714 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:42.049718 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:42.049767 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:42.049866 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:42.050242 (+ 376us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40786: BlockingRecv error: recv got EOF from 127.0.0.1:40786 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
W20260929 10:37:42.061475 5634 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:37:42.060115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40802 (local address 127.15.82.125:42085)
0929 10:37:42.060258 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:42.060263 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:42.060563 (+ 300us) server_negotiation.cc:407] Connection header received
0929 10:37:42.060736 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:42.060739 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:42.060802 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:42.060899 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:42.061326 (+ 427us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40802: BlockingRecv error: recv got EOF from 127.0.0.1:40802 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260929 10:37:42.081023 5634 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:37:42.079856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40806 (local address 127.15.82.125:42085)
0929 10:37:42.080023 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:42.080031 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:42.080146 (+ 115us) server_negotiation.cc:407] Connection header received
0929 10:37:42.080252 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:42.080255 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:42.080321 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:42.080433 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:42.080893 (+ 460us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40806: BlockingRecv error: recv got EOF from 127.0.0.1:40806 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260929 10:37:42.116588 5634 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:37:42.115180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40820 (local address 127.15.82.125:42085)
0929 10:37:42.115339 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:42.115342 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:42.115539 (+ 197us) server_negotiation.cc:407] Connection header received
0929 10:37:42.115637 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:42.115640 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:42.115693 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:42.115777 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:42.116442 (+ 665us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40820: BlockingRecv error: recv got EOF from 127.0.0.1:40820 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260929 10:37:42.184019 5634 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:37:42.182782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40822 (local address 127.15.82.125:42085)
0929 10:37:42.182948 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:42.182951 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:42.183054 (+ 103us) server_negotiation.cc:407] Connection header received
0929 10:37:42.183223 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:42.183226 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:42.183293 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:42.183382 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:42.183903 (+ 521us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40822: BlockingRecv error: recv got EOF from 127.0.0.1:40822 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
W20260929 10:37:42.315634 5634 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:37:42.314273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40832 (local address 127.15.82.125:42085)
0929 10:37:42.314491 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:42.314496 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:42.314638 (+ 142us) server_negotiation.cc:407] Connection header received
0929 10:37:42.314845 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:42.314860 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:42.314912 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:42.315010 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:42.315505 (+ 495us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40832: BlockingRecv error: recv got EOF from 127.0.0.1:40832 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
W20260929 10:37:42.575197 5634 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:37:42.573885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40848 (local address 127.15.82.125:42085)
0929 10:37:42.574052 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:42.574058 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:42.574153 (+ 95us) server_negotiation.cc:407] Connection header received
0929 10:37:42.574337 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:42.574341 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:42.574467 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:42.574576 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:42.575079 (+ 503us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40848: BlockingRecv error: recv got EOF from 127.0.0.1:40848 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
I20260929 10:37:42.737972 5723 catalog_manager.cc:1259] Loaded cluster ID: 54ef35191d4e46c4a9cd436e467b6f26
I20260929 10:37:42.738068 5723 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 688147f3226f407da540afc3e91f4a09: loading cluster ID for follower catalog manager: success
I20260929 10:37:42.739831 5723 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 688147f3226f407da540afc3e91f4a09: acquiring CA information for follower catalog manager: success
I20260929 10:37:42.741103 5723 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 688147f3226f407da540afc3e91f4a09: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 10:37:42.768369 5737 catalog_manager.cc:1259] Loaded cluster ID: 54ef35191d4e46c4a9cd436e467b6f26
I20260929 10:37:42.768508 5737 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e3ea36bd2db64eeb803c460a8ebd4a69: loading cluster ID for follower catalog manager: success
I20260929 10:37:42.770247 5737 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e3ea36bd2db64eeb803c460a8ebd4a69: acquiring CA information for follower catalog manager: success
I20260929 10:37:42.771584 5737 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e3ea36bd2db64eeb803c460a8ebd4a69: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 10:37:43.093869 5837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:43.089567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40854 (local address 127.15.82.125:42085)
0929 10:37:43.089864 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:43.089869 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:43.089889 (+ 20us) server_negotiation.cc:407] Connection header received
0929 10:37:43.090092 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:43.090094 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:43.090227 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:43.090361 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:37:43.090716 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:43.091304 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:43.092210 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:43.092455 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:43.092581 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:43.092940 (+ 359us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:37:43.093407 (+ 467us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:37:43.093567 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:43.093608 (+ 41us) server_negotiation.cc:299] Negotiation successful
0929 10:37:43.093671 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":76,"threads_started":1}
I20260929 10:37:43.101150 5843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:43.097196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55274 (local address 127.15.82.124:42611)
0929 10:37:43.097483 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:43.097487 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:43.097503 (+ 16us) server_negotiation.cc:407] Connection header received
0929 10:37:43.097667 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:43.097669 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:43.097812 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:43.097920 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:37:43.098112 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:43.098771 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:43.099606 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:43.099852 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:43.099943 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:43.100213 (+ 270us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:37:43.100655 (+ 442us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:37:43.100766 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:43.100867 (+ 101us) server_negotiation.cc:299] Negotiation successful
0929 10:37:43.100921 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":86,"threads_started":1}
I20260929 10:37:43.109421 5843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:43.104986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55276 (local address 127.15.82.124:42611)
0929 10:37:43.105324 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:43.105327 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:43.105337 (+ 10us) server_negotiation.cc:407] Connection header received
0929 10:37:43.105372 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:43.105373 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:43.105410 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:43.105494 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:37:43.105854 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:43.106641 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:43.107939 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:43.108315 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:43.108419 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:43.108707 (+ 288us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:37:43.108802 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:37:43.108876 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:43.109221 (+ 345us) server_negotiation.cc:299] Negotiation successful
0929 10:37:43.109293 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226}
I20260929 10:37:43.109619 5837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:43.104605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40858 (local address 127.15.82.125:42085)
0929 10:37:43.104739 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:43.104745 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:43.105226 (+ 481us) server_negotiation.cc:407] Connection header received
0929 10:37:43.105300 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:43.105303 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:43.105357 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:43.105473 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:37:43.105715 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:43.106389 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:43.107912 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:43.108249 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:43.108353 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:43.108777 (+ 424us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:37:43.108934 (+ 157us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:37:43.109024 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:43.109197 (+ 173us) server_negotiation.cc:299] Negotiation successful
0929 10:37:43.109485 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260929 10:37:43.110813 5850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:43.104571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44758 (local address 127.15.82.126:38805)
0929 10:37:43.104822 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:43.104826 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:43.105134 (+ 308us) server_negotiation.cc:407] Connection header received
0929 10:37:43.105262 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:43.105266 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:43.105463 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:43.105584 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:37:43.106001 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:43.106819 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:43.107660 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:43.107961 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:43.108055 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:43.109608 (+ 1553us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:37:43.110436 (+ 828us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:37:43.110530 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:43.110640 (+ 110us) server_negotiation.cc:299] Negotiation successful
0929 10:37:43.110695 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":81,"threads_started":1}
I20260929 10:37:43.114320 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 5485
I20260929 10:37:43.121178 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 5560
I20260929 10:37:43.127511 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 5636
2026-09-29T10:37:43Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3261 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790677981592896-15689-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:37:43 dist-test-slave-vg89 krb5kdc[5854](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:37:43 dist-test-slave-vg89 krb5kdc[5854](info): set up 2 sockets
Sep 29 10:37:43 dist-test-slave-vg89 krb5kdc[5854](info): commencing operation
krb5kdc: starting...
W20260929 10:37:44.195169 15689 mini_kdc.cc:121] Time spent starting KDC: real 1.045s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790677981592896-15689-0/krb5kdc/test-user.keytab.
Sep 29 10:37:44 dist-test-slave-vg89 krb5kdc[5854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678264, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:37:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:37:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.82.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab.
Entry for principal kudu/127.15.82.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab.
I20260929 10:37:44.321141 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790677981592896-15689-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790677981592896-15689-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790677981592896-15689-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:45357
--webserver_interface=127.15.82.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab
--principal=kudu/127.15.82.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:46435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.82.126:45357,127.15.82.125:38327,127.15.82.124:44201
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:37:44.436611 5867 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 10:37:44.437007 5867 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:37:44.437072 5867 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:37:44.437153 5867 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:37:44.441418 5867 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:37:44.441529 5867 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:37:44.441566 5867 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:37:44.441602 5867 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:37:44.441641 5867 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:37:44.447310 5867 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:46435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790677981592896-15689-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790677981592896-15689-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.82.126:45357,127.15.82.125:38327,127.15.82.124:44201
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.126.keytab
--principal=kudu/127.15.82.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.126:45357
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.82.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5867
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790677981592896-15689-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:37:44.448511 5867 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:37:44.449630 5867 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 10:37:44.458729 5867 server_base.cc:1034] running on GCE node
W20260929 10:37:44.458676 5873 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:44.458676 5872 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:44.458680 5875 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:37:44.459698 5867 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:37:44.460851 5867 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:37:44.462030 5867 hybrid_clock.cc:648] HybridClock initialized: now 1790678264462014 us; error 27 us; skew 500 ppm
Sep 29 10:37:44 dist-test-slave-vg89 krb5kdc[5854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678264, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:37:44.465193 5867 init.cc:377] Logged in from keytab as kudu/127.15.82.126@KRBTEST.COM (short username kudu)
I20260929 10:37:44.466151 5867 webserver.cc:460] Webserver started at http://127.15.82.126:40315/ using document root <none> and password file <none>
I20260929 10:37:44.466807 5867 fs_manager.cc:359] Metadata directory not provided
I20260929 10:37:44.466908 5867 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:37:44.467155 5867 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:37:44.469360 5867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790677981592896-15689-0/minicluster-data/master-0/data/instance:
uuid: "70255adb66914018a8b1a4addb520a38"
format_stamp: "Formatted at 2026-09-29 10:37:44 on dist-test-slave-vg89"
server_key: "26f24dd614e30893d0d15adb6d500798"
server_key_iv: "d4c6807d939716d75c6ba65648796bba"
server_key_version: "encryptionkey@0"
I20260929 10:37:44.469987 5867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790677981592896-15689-0/minicluster-data/master-0/wal/instance:
uuid: "70255adb66914018a8b1a4addb520a38"
format_stamp: "Formatted at 2026-09-29 10:37:44 on dist-test-slave-vg89"
server_key: "26f24dd614e30893d0d15adb6d500798"
server_key_iv: "d4c6807d939716d75c6ba65648796bba"
server_key_version: "encryptionkey@0"
I20260929 10:37:44.474098 5867 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260929 10:37:44.476867 5882 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:44.478354 5867 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 10:37:44.478591 5867 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790677981592896-15689-0/minicluster-data/master-0/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790677981592896-15689-0/minicluster-data/master-0/wal
uuid: "70255adb66914018a8b1a4addb520a38"
format_stamp: "Formatted at 2026-09-29 10:37:44 on dist-test-slave-vg89"
server_key: "26f24dd614e30893d0d15adb6d500798"
server_key_iv: "d4c6807d939716d75c6ba65648796bba"
server_key_version: "encryptionkey@0"
I20260929 10:37:44.478731 5867 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790677981592896-15689-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790677981592896-15689-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790677981592896-15689-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:37:44.504110 5867 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:37:44.507687 5867 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:37:44.507908 5867 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:37:44.515268 5867 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.126:45357
I20260929 10:37:44.515584 5934 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.126:45357 every 8 connection(s)
I20260929 10:37:44.516299 5867 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790677981592896-15689-0/minicluster-data/master-0/data/info.pb
I20260929 10:37:44.518464 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 5867
I20260929 10:37:44.518617 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790677981592896-15689-0/minicluster-data/master-0/wal/instance
I20260929 10:37:44.518918 15689 external_mini_cluster.cc:1428] Setting key 0cd867fc3ec922b9fafb70f1477a2db2
I20260929 10:37:44.519431 5935 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:37:44.525195 5935 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.82.126" port: 45357 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.15.82.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.125@KRBTEST.COM" created.
Sep 29 10:37:44 dist-test-slave-vg89 krb5kdc[5854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678264, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.126@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:37:44.542117 5938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:44.528257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:45357 (local address 127.0.0.1:41788)
0929 10:37:44.528850 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:44.528902 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0929 10:37:44.529855 (+ 953us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:37:44.530728 (+ 873us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:37:44.530735 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:37:44.531175 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:37:44.531422 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:44.531436 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:44.532688 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:44.532691 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:44.533431 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:44.533441 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:44.534135 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:44.534139 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:44.534239 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:44.534959 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:37:44.534976 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:37:44.536724 (+ 1748us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:37:44.538875 (+ 2151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:37:44.538884 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:37:44.538894 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:37:44.539130 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:37:44.539458 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:37:44.539463 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:37:44.539465 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:37:44.539582 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:37:44.540241 (+ 659us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:37:44.540249 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:37:44.540408 (+ 159us) client_negotiation.cc:769] Sending connection context
0929 10:37:44.540626 (+ 218us) client_negotiation.cc:240] Negotiation successful
0929 10:37:44.541050 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"mutex_wait_us":105,"thread_start_us":163,"threads_started":1}
I20260929 10:37:44.542117 5939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:44.528256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41788 (local address 127.15.82.126:45357)
0929 10:37:44.528832 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:44.528851 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:44.529021 (+ 170us) server_negotiation.cc:407] Connection header received
0929 10:37:44.530184 (+ 1163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:44.530193 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:44.530486 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:44.530687 (+ 201us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:44.531579 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:44.532528 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:44.533572 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:44.533956 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:44.534088 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:44.536899 (+ 2811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:44.536904 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:44.536915 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:37:44.536936 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:44.538733 (+ 1797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:44.539245 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:44.539251 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:44.539253 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:44.539364 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:44.539679 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:44.539682 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:44.539684 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:44.540126 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:44.540275 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:44.540687 (+ 412us) server_negotiation.cc:299] Negotiation successful
0929 10:37:44.540971 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":450,"spinlock_wait_cycles":55680,"thread_start_us":191,"threads_started":1}
I20260929 10:37:44.545089 5935 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.82.125" port: 38327 } has no permanent_uuid. Determining permanent_uuid...
I20260929 10:37:44.546033 5938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:44.545668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.125:38327 (local address 127.0.0.1:57856)
0929 10:37:44.545789 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:44.545900 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.125:38327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260929 10:37:44.546263 5885 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.82.125:38327: connect: Connection refused (error 111)
I20260929 10:37:44.547286 5938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:44.547045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.125:38327 (local address 127.0.0.1:57872)
0929 10:37:44.547149 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:44.547197 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.125:38327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260929 10:37:44.547505 5935 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.82.125:38327: Network error: Client connection negotiation failed: client connection to 127.15.82.125:38327: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.125.keytab.
Entry for principal kudu/127.15.82.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.125.keytab.
I20260929 10:37:44.550849 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790677981592896-15689-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790677981592896-15689-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790677981592896-15689-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790677981592896-15689-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.125:38327
--webserver_interface=127.15.82.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.125.keytab
--principal=kudu/127.15.82.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:46435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.82.126:45357,127.15.82.125:38327,127.15.82.124:44201
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 10:37:44.596764 5935 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.82.125" port: 38327 } attempt: 1
I20260929 10:37:44.598006 5938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:44.597592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.125:38327 (local address 127.0.0.1:57886)
0929 10:37:44.597780 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:44.597898 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.125:38327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260929 10:37:44.598887 5938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:44.598643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.125:38327 (local address 127.0.0.1:57900)
0929 10:37:44.598759 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:44.598820 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.125:38327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260929 10:37:44.599047 5935 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.82.125:38327: Network error: Client connection negotiation failed: client connection to 127.15.82.125:38327: connect: Connection refused (error 111)
W20260929 10:37:44.665503 5942 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 10:37:44.665871 5942 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:37:44.665933 5942 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:37:44.666013 5942 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260929 10:37:44.667222 5935 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.82.125" port: 38327 } attempt: 2
I20260929 10:37:44.668089 5938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:44.667749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.125:38327 (local address 127.0.0.1:57912)
0929 10:37:44.667923 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:44.667996 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.125:38327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260929 10:37:44.668905 5938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:44.668674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.125:38327 (local address 127.0.0.1:57922)
0929 10:37:44.668783 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:44.668839 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.125:38327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260929 10:37:44.669067 5935 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.82.125:38327: Network error: Client connection negotiation failed: client connection to 127.15.82.125:38327: connect: Connection refused (error 111)
W20260929 10:37:44.670739 5942 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:37:44.670789 5942 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:37:44.670816 5942 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:37:44.670872 5942 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:37:44.670897 5942 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:37:44.676383 5942 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:46435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790677981592896-15689-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790677981592896-15689-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.15.82.126:45357,127.15.82.125:38327,127.15.82.124:44201
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.125.keytab
--principal=kudu/127.15.82.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.125:38327
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790677981592896-15689-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.15.82.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5942
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790677981592896-15689-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:37:44.677525 5942 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:37:44.678678 5942 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:37:44.687355 5948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:44.687345 5950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:44.687367 5947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:37:44.687619 5942 server_base.cc:1034] running on GCE node
I20260929 10:37:44.688154 5942 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:37:44.689323 5942 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:37:44.690477 5942 hybrid_clock.cc:648] HybridClock initialized: now 1790678264690459 us; error 36 us; skew 500 ppm
Sep 29 10:37:44 dist-test-slave-vg89 krb5kdc[5854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678264, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:37:44.693552 5942 init.cc:377] Logged in from keytab as kudu/127.15.82.125@KRBTEST.COM (short username kudu)
I20260929 10:37:44.694571 5942 webserver.cc:460] Webserver started at http://127.15.82.125:37861/ using document root <none> and password file <none>
I20260929 10:37:44.695161 5942 fs_manager.cc:359] Metadata directory not provided
I20260929 10:37:44.695240 5942 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:37:44.695427 5942 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:37:44.697741 5942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790677981592896-15689-0/minicluster-data/master-1/data/instance:
uuid: "a992d3ad0daa4078b50343e6463575a8"
format_stamp: "Formatted at 2026-09-29 10:37:44 on dist-test-slave-vg89"
server_key: "9deff39d946d3364864407cf1ef99ae5"
server_key_iv: "63e0fa92184c1106e2e31ebba4a0758a"
server_key_version: "encryptionkey@0"
I20260929 10:37:44.698340 5942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790677981592896-15689-0/minicluster-data/master-1/wal/instance:
uuid: "a992d3ad0daa4078b50343e6463575a8"
format_stamp: "Formatted at 2026-09-29 10:37:44 on dist-test-slave-vg89"
server_key: "9deff39d946d3364864407cf1ef99ae5"
server_key_iv: "63e0fa92184c1106e2e31ebba4a0758a"
server_key_version: "encryptionkey@0"
I20260929 10:37:44.702507 5942 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260929 10:37:44.704994 5957 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:44.706231 5942 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 10:37:44.706332 5942 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790677981592896-15689-0/minicluster-data/master-1/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790677981592896-15689-0/minicluster-data/master-1/wal
uuid: "a992d3ad0daa4078b50343e6463575a8"
format_stamp: "Formatted at 2026-09-29 10:37:44 on dist-test-slave-vg89"
server_key: "9deff39d946d3364864407cf1ef99ae5"
server_key_iv: "63e0fa92184c1106e2e31ebba4a0758a"
server_key_version: "encryptionkey@0"
I20260929 10:37:44.706499 5942 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790677981592896-15689-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790677981592896-15689-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790677981592896-15689-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:37:44.719980 5942 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:37:44.722985 5942 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:37:44.723132 5942 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:37:44.730180 5942 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.125:38327
I20260929 10:37:44.730516 6009 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.125:38327 every 8 connection(s)
I20260929 10:37:44.731381 5942 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790677981592896-15689-0/minicluster-data/master-1/data/info.pb
I20260929 10:37:44.734598 6010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:37:44.738193 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 5942
I20260929 10:37:44.738361 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790677981592896-15689-0/minicluster-data/master-1/wal/instance
I20260929 10:37:44.738686 15689 external_mini_cluster.cc:1428] Setting key b7c5d9b7be47194eac6e2de534d3b0cf
I20260929 10:37:44.738435 6010 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.82.126" port: 45357 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.15.82.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.82.124@KRBTEST.COM" created.
Sep 29 10:37:44 dist-test-slave-vg89 krb5kdc[5854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678264, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.125@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
I20260929 10:37:44.755762 5939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:44.740381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41802 (local address 127.15.82.126:45357)
0929 10:37:44.740511 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:44.740515 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:44.741454 (+ 939us) server_negotiation.cc:407] Connection header received
0929 10:37:44.742428 (+ 974us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:44.742435 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:44.742504 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:44.742610 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:44.743881 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:44.744820 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:44.746072 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:44.746534 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:44.746664 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:44.751707 (+ 5043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:44.751711 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:44.751713 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:37:44.751741 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:44.753269 (+ 1528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:44.753818 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:44.753825 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:44.753830 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:44.753928 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:44.754260 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:44.754266 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:44.754270 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:44.754611 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:44.754728 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:44.755366 (+ 638us) server_negotiation.cc:299] Negotiation successful
0929 10:37:44.755515 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260929 10:37:44.756552 6013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:44.740793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:45357 (local address 127.0.0.1:41802)
0929 10:37:44.741257 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:44.741320 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0929 10:37:44.742170 (+ 850us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:37:44.742770 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:37:44.742777 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:37:44.743216 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:37:44.743707 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:44.743720 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:44.744993 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:44.744996 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:44.745935 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:44.745945 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:44.746687 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:44.746691 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:44.746879 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:44.747589 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:37:44.747606 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:37:44.751496 (+ 3890us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:37:44.753428 (+ 1932us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:37:44.753433 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:37:44.753447 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:37:44.753711 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:37:44.754024 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:37:44.754026 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:37:44.754028 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:37:44.754165 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:37:44.754722 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:37:44.754727 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:37:44.755085 (+ 358us) client_negotiation.cc:769] Sending connection context
0929 10:37:44.755257 (+ 172us) client_negotiation.cc:240] Negotiation successful
0929 10:37:44.755532 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":143,"threads_started":1}
I20260929 10:37:44.758208 6010 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.82.125" port: 38327 } has no permanent_uuid. Determining permanent_uuid...
I20260929 10:37:44.760262 5935 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.82.125" port: 38327 } attempt: 3
Sep 29 10:37:44 dist-test-slave-vg89 krb5kdc[5854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678264, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.125@KRBTEST.COM for kudu/127.15.82.125@KRBTEST.COM
Sep 29 10:37:44 dist-test-slave-vg89 krb5kdc[5854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678264, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.126@KRBTEST.COM for kudu/127.15.82.125@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.82.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.124.keytab.
Entry for principal kudu/127.15.82.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.124.keytab.
I20260929 10:37:44.769143 6013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:44.758882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.125:38327 (local address 127.0.0.1:57928)
0929 10:37:44.759000 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:44.759015 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 10:37:44.759096 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:37:44.759660 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:37:44.759663 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:37:44.759845 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:37:44.759937 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:44.759943 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:44.761342 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:44.761346 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:44.762055 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:44.762063 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:44.762708 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:44.762712 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:44.762829 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:44.763384 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:37:44.763395 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:37:44.764887 (+ 1492us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:37:44.767131 (+ 2244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:37:44.767134 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:37:44.767135 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:37:44.767369 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:37:44.767796 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:37:44.767798 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:37:44.767800 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:37:44.767877 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:37:44.768520 (+ 643us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:37:44.768522 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:37:44.768597 (+ 75us) client_negotiation.cc:769] Sending connection context
0929 10:37:44.768721 (+ 124us) client_negotiation.cc:240] Negotiation successful
0929 10:37:44.768873 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260929 10:37:44.769325 6015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:44.758935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57928 (local address 127.15.82.125:38327)
0929 10:37:44.759214 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:44.759226 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:44.759246 (+ 20us) server_negotiation.cc:407] Connection header received
0929 10:37:44.759276 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:44.759280 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:44.759514 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:44.759660 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:44.760044 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:44.761236 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:44.762164 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:44.762595 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:44.762678 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:44.765034 (+ 2356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:44.765042 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:44.765069 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:37:44.765089 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:44.767031 (+ 1942us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:44.767512 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:44.767517 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:44.767518 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:44.767705 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:44.767972 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:44.767977 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:44.767980 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:44.768414 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:44.768589 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:44.768916 (+ 327us) server_negotiation.cc:299] Negotiation successful
0929 10:37:44.769100 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":89,"threads_started":1}
I20260929 10:37:44.770004 15689 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
/tmp/dist-test-taskiKMUyb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790677981592896-15689-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790677981592896-15689-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790677981592896-15689-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790677981592896-15689-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.124:44201
--webserver_interface=127.15.82.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.124.keytab
--principal=kudu/127.15.82.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.82.84:46435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.82.126:45357,127.15.82.125:38327,127.15.82.124:44201
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790677981592896-15689-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790677981592896-15689-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790677981592896-15689-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 10:37:44.771545 6010 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.82.124" port: 44201 } has no permanent_uuid. Determining permanent_uuid...
I20260929 10:37:44.772279 6013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:44.772026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.124:44201 (local address 127.0.0.1:49494)
0929 10:37:44.772144 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:44.772221 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.124:44201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260929 10:37:44.772469 5960 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.82.124:44201: connect: Connection refused (error 111)
I20260929 10:37:44.773471 6013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:44.773226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.124:44201 (local address 127.0.0.1:49498)
0929 10:37:44.773319 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:44.773375 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.124:44201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260929 10:37:44.773768 6010 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.82.124:44201: Network error: Client connection negotiation failed: client connection to 127.15.82.124:44201: connect: Connection refused (error 111)
I20260929 10:37:44.774245 6016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:44.760890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57944 (local address 127.15.82.125:38327)
0929 10:37:44.761277 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:44.761285 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:44.761315 (+ 30us) server_negotiation.cc:407] Connection header received
0929 10:37:44.761344 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:44.761347 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:44.761391 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:44.761481 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:44.761927 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:44.762714 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:44.764198 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:44.764484 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:44.764613 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:44.767135 (+ 2522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:44.767139 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:44.767142 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:37:44.767165 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:44.772370 (+ 5205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:44.773020 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:44.773023 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:44.773025 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:44.773110 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:44.773414 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:44.773418 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:44.773419 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:44.773638 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:44.773739 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:44.774038 (+ 299us) server_negotiation.cc:299] Negotiation successful
0929 10:37:44.774128 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":151,"threads_started":1}
I20260929 10:37:44.774459 5938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:44.760641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.125:38327 (local address 127.0.0.1:57944)
0929 10:37:44.760901 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:44.760918 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 10:37:44.761010 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:37:44.761553 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:37:44.761556 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:37:44.761691 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:37:44.761786 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:44.761791 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:44.762836 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:44.762838 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:44.764078 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:44.764089 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:44.764625 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:44.764629 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:44.764744 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:44.765306 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:37:44.765326 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:37:44.766958 (+ 1632us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:37:44.772558 (+ 5600us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:37:44.772562 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:37:44.772564 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:37:44.772895 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:37:44.773211 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:37:44.773214 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:37:44.773215 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:37:44.773307 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:37:44.773734 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:37:44.773740 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:37:44.773873 (+ 133us) client_negotiation.cc:769] Sending connection context
0929 10:37:44.773995 (+ 122us) client_negotiation.cc:240] Negotiation successful
0929 10:37:44.774166 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101}
I20260929 10:37:44.775265 5935 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.82.124" port: 44201 } has no permanent_uuid. Determining permanent_uuid...
I20260929 10:37:44.776081 5938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:44.775793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.124:44201 (local address 127.0.0.1:49502)
0929 10:37:44.775956 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:44.776003 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.124:44201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260929 10:37:44.776919 5938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:44.776677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.124:44201 (local address 127.0.0.1:49518)
0929 10:37:44.776823 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:44.776874 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.124:44201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
W20260929 10:37:44.777120 5935 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.82.124:44201: Network error: Client connection negotiation failed: client connection to 127.15.82.124:44201: connect: Connection refused (error 111)
I20260929 10:37:44.808319 5935 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.82.124" port: 44201 } attempt: 1
I20260929 10:37:44.809392 5938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:44.808984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.124:44201 (local address 127.0.0.1:49534)
0929 10:37:44.809172 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:44.809299 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.124:44201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260929 10:37:44.810271 5938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:44.810029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.124:44201 (local address 127.0.0.1:49544)
0929 10:37:44.810161 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:44.810217 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.124:44201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260929 10:37:44.810606 5935 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.82.124:44201: Network error: Client connection negotiation failed: client connection to 127.15.82.124:44201: connect: Connection refused (error 111)
I20260929 10:37:44.823025 6010 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.82.124" port: 44201 } attempt: 1
I20260929 10:37:44.823832 6013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:44.823542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.124:44201 (local address 127.0.0.1:49554)
0929 10:37:44.823680 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:44.823748 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.124:44201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260929 10:37:44.824590 6013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:44.824372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.124:44201 (local address 127.0.0.1:49566)
0929 10:37:44.824482 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:44.824535 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.124:44201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260929 10:37:44.824719 6010 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.82.124:44201: Network error: Client connection negotiation failed: client connection to 127.15.82.124:44201: connect: Connection refused (error 111)
W20260929 10:37:44.884387 6017 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 10:37:44.884716 6017 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:37:44.884788 6017 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:37:44.884878 6017 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260929 10:37:44.885833 5935 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.82.124" port: 44201 } attempt: 2
I20260929 10:37:44.886677 5938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:44.886367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.124:44201 (local address 127.0.0.1:49582)
0929 10:37:44.886535 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:44.886597 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.124:44201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260929 10:37:44.887405 5938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:44.887216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.124:44201 (local address 127.0.0.1:49588)
0929 10:37:44.887316 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:44.887358 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.124:44201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260929 10:37:44.887532 5935 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.82.124:44201: Network error: Client connection negotiation failed: client connection to 127.15.82.124:44201: connect: Connection refused (error 111)
W20260929 10:37:44.889598 6017 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:37:44.889679 6017 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:37:44.889724 6017 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:37:44.889757 6017 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:37:44.889798 6017 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:37:44.892887 6010 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.82.124" port: 44201 } attempt: 2
I20260929 10:37:44.893584 6013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:44.893329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.124:44201 (local address 127.0.0.1:49590)
0929 10:37:44.893456 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:44.893518 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.124:44201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260929 10:37:44.894299 6013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:44.894073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.124:44201 (local address 127.0.0.1:49602)
0929 10:37:44.894182 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:44.894231 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.82.124:44201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260929 10:37:44.894449 6010 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.82.124:44201: Network error: Client connection negotiation failed: client connection to 127.15.82.124:44201: connect: Connection refused (error 111)
I20260929 10:37:44.895514 6017 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.82.84:46435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790677981592896-15689-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790677981592896-15689-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.15.82.126:45357,127.15.82.125:38327,127.15.82.124:44201
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790677981592896-15689-0/krb5kdc/kudu_127.15.82.124.keytab
--principal=kudu/127.15.82.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.82.124:44201
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790677981592896-15689-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.15.82.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6017
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790677981592896-15689-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 09:44:11 UTC on ccycloud.quasar-jhrwyf.root.comops.site
build id 4679897
I20260929 10:37:44.896615 6017 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:37:44.897648 6017 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:37:44.906158 6026 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:37:44.906188 6023 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:37:44.906337 6017 server_base.cc:1034] running on GCE node
W20260929 10:37:44.906158 6024 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:37:44.907130 6017 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:37:44.908324 6017 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:37:44.909533 6017 hybrid_clock.cc:648] HybridClock initialized: now 1790678264909515 us; error 25 us; skew 500 ppm
Sep 29 10:37:44 dist-test-slave-vg89 krb5kdc[5854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678264, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:37:44.912609 6017 init.cc:377] Logged in from keytab as kudu/127.15.82.124@KRBTEST.COM (short username kudu)
I20260929 10:37:44.913551 6017 webserver.cc:460] Webserver started at http://127.15.82.124:41501/ using document root <none> and password file <none>
I20260929 10:37:44.914151 6017 fs_manager.cc:359] Metadata directory not provided
I20260929 10:37:44.914251 6017 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:37:44.914561 6017 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:37:44.916788 6017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790677981592896-15689-0/minicluster-data/master-2/data/instance:
uuid: "d4d7a827cb594bfead942e2d659eb116"
format_stamp: "Formatted at 2026-09-29 10:37:44 on dist-test-slave-vg89"
server_key: "e6a9e0b6ebb460d70f433210a4c4cd84"
server_key_iv: "4e58be1d31a153aae611814a89fd876f"
server_key_version: "encryptionkey@0"
I20260929 10:37:44.917488 6017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790677981592896-15689-0/minicluster-data/master-2/wal/instance:
uuid: "d4d7a827cb594bfead942e2d659eb116"
format_stamp: "Formatted at 2026-09-29 10:37:44 on dist-test-slave-vg89"
server_key: "e6a9e0b6ebb460d70f433210a4c4cd84"
server_key_iv: "4e58be1d31a153aae611814a89fd876f"
server_key_version: "encryptionkey@0"
I20260929 10:37:44.921689 6017 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260929 10:37:44.924450 6033 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:37:44.925822 6017 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260929 10:37:44.925993 6017 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790677981592896-15689-0/minicluster-data/master-2/data,/tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790677981592896-15689-0/minicluster-data/master-2/wal
uuid: "d4d7a827cb594bfead942e2d659eb116"
format_stamp: "Formatted at 2026-09-29 10:37:44 on dist-test-slave-vg89"
server_key: "e6a9e0b6ebb460d70f433210a4c4cd84"
server_key_iv: "4e58be1d31a153aae611814a89fd876f"
server_key_version: "encryptionkey@0"
I20260929 10:37:44.926120 6017 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790677981592896-15689-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790677981592896-15689-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790677981592896-15689-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:37:44.948887 6017 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:37:44.952121 6017 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:37:44.952338 6017 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:37:44.959272 6017 rpc_server.cc:307] RPC server started. Bound to: 127.15.82.124:44201
I20260929 10:37:44.959551 6085 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.82.124:44201 every 8 connection(s)
I20260929 10:37:44.960300 6017 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790677981592896-15689-0/minicluster-data/master-2/data/info.pb
I20260929 10:37:44.963229 6086 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:37:44.967062 6086 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.82.126" port: 45357 } has no permanent_uuid. Determining permanent_uuid...
I20260929 10:37:44.968326 15689 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu as pid 6017
I20260929 10:37:44.968509 15689 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiKMUyb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790677981592896-15689-0/minicluster-data/master-2/wal/instance
I20260929 10:37:44.968854 15689 external_mini_cluster.cc:1428] Setting key cc83ca9cc19e4afd2569183a8eeee7ae
Sep 29 10:37:44 dist-test-slave-vg89 krb5kdc[5854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678264, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.124@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
Sep 29 10:37:44 dist-test-slave-vg89 krb5kdc[5854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678264, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.82.124@KRBTEST.COM
I20260929 10:37:44.984819 5939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:44.968999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41806 (local address 127.15.82.126:45357)
0929 10:37:44.969173 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:44.969179 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:44.970154 (+ 975us) server_negotiation.cc:407] Connection header received
0929 10:37:44.971225 (+ 1071us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:44.971228 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:44.971302 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:44.971384 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:44.972560 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:44.973342 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:44.974941 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:44.975284 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:44.975431 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:44.979635 (+ 4204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:44.979640 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:44.979644 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:37:44.979683 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:44.981839 (+ 2156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:44.982633 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:44.982638 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:44.982642 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:44.982795 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:44.983306 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:44.983309 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:44.983312 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:44.983599 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:44.983702 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:44.984394 (+ 692us) server_negotiation.cc:299] Negotiation successful
0929 10:37:44.984550 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260929 10:37:44.985612 6010 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.82.124" port: 44201 } attempt: 3
I20260929 10:37:44.985733 6090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:44.969602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49608 (local address 127.15.82.124:44201)
0929 10:37:44.969988 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:44.970005 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:44.970218 (+ 213us) server_negotiation.cc:407] Connection header received
0929 10:37:44.970973 (+ 755us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:44.970989 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:44.971233 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:44.971420 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:44.972394 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:44.973650 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:44.974949 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:44.975509 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:44.975640 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:44.980115 (+ 4475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:44.980125 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:44.980144 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:37:44.980176 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:44.982520 (+ 2344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:44.983021 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:44.983027 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:44.983029 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:44.983142 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:44.983446 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:44.983449 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:44.983451 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:44.983827 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:44.984036 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:44.984341 (+ 305us) server_negotiation.cc:299] Negotiation successful
0929 10:37:44.984657 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":282,"spinlock_wait_cycles":3968,"thread_start_us":185,"threads_started":1}
I20260929 10:37:44.986272 6088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:44.969374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.126:45357 (local address 127.0.0.1:41806)
0929 10:37:44.969985 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:44.970042 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0929 10:37:44.970992 (+ 950us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:37:44.971462 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:37:44.971469 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:37:44.972009 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:37:44.972425 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:44.972440 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:44.973454 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:44.973457 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:44.974727 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:44.974739 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:44.975468 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:44.975471 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:44.975646 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:44.976502 (+ 856us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:37:44.976529 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:37:44.979470 (+ 2941us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:37:44.981947 (+ 2477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:37:44.981958 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:37:44.981989 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:37:44.982483 (+ 494us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:37:44.982933 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:37:44.982937 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:37:44.982940 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:37:44.983151 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:37:44.983776 (+ 625us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:37:44.983785 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:37:44.983969 (+ 184us) client_negotiation.cc:769] Sending connection context
0929 10:37:44.985067 (+ 1098us) client_negotiation.cc:240] Negotiation successful
0929 10:37:44.985302 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"mutex_wait_us":24,"spinlock_wait_cycles":1280,"thread_start_us":167,"threads_started":1}
I20260929 10:37:44.987032 5935 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.82.124" port: 44201 } attempt: 3
I20260929 10:37:44.987643 6086 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.82.125" port: 38327 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 10:37:44 dist-test-slave-vg89 krb5kdc[5854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678264, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.125@KRBTEST.COM for kudu/127.15.82.124@KRBTEST.COM
Sep 29 10:37:44 dist-test-slave-vg89 krb5kdc[5854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678264, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.126@KRBTEST.COM for kudu/127.15.82.124@KRBTEST.COM
Sep 29 10:37:44 dist-test-slave-vg89 krb5kdc[5854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678264, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.82.126@KRBTEST.COM
Sep 29 10:37:44 dist-test-slave-vg89 krb5kdc[5854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678264, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.124@KRBTEST.COM for kudu/127.15.82.125@KRBTEST.COM
I20260929 10:37:44.997817 6013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:44.986083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.124:44201 (local address 127.0.0.1:49618)
0929 10:37:44.986242 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:44.986257 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 10:37:44.986339 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:37:44.987367 (+ 1028us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:37:44.987370 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:37:44.987555 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:37:44.987665 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:44.987674 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:44.988744 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:44.988747 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:44.989543 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:44.989553 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:44.990152 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:44.990156 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:44.991235 (+ 1079us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:44.991832 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:37:44.991848 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:37:44.993986 (+ 2138us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:37:44.996218 (+ 2232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:37:44.996222 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:37:44.996224 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:37:44.996432 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:37:44.996710 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:37:44.996714 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:37:44.996716 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:37:44.996814 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:37:44.997268 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:37:44.997271 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:37:44.997364 (+ 93us) client_negotiation.cc:769] Sending connection context
0929 10:37:44.997471 (+ 107us) client_negotiation.cc:240] Negotiation successful
0929 10:37:44.997614 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61,"mutex_wait_us":983,"spinlock_wait_cycles":4992}
I20260929 10:37:44.998153 6090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:44.987070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49618 (local address 127.15.82.124:44201)
0929 10:37:44.987159 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:44.987163 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:44.987173 (+ 10us) server_negotiation.cc:407] Connection header received
0929 10:37:44.987202 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:44.987204 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:44.987243 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:44.987786 (+ 543us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:44.987865 (+ 79us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:44.988601 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:44.989673 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:44.990039 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:44.990262 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:44.994110 (+ 3848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:44.994116 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:44.994123 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:37:44.994158 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:44.996108 (+ 1950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:44.996531 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:44.996535 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:44.996537 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:44.996619 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:44.996925 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:44.996929 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:44.996932 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:44.997119 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:44.997237 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:44.997521 (+ 284us) server_negotiation.cc:299] Negotiation successful
0929 10:37:44.997627 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":26}
I20260929 10:37:45.000835 5939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:44.989330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41812 (local address 127.15.82.126:45357)
0929 10:37:44.989439 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:44.989444 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:44.989456 (+ 12us) server_negotiation.cc:407] Connection header received
0929 10:37:44.989490 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:44.989495 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:44.989538 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:44.989656 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:44.990499 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:44.991308 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:44.992588 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:44.992968 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:44.993072 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:44.995908 (+ 2836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:44.995912 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:44.995915 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:37:44.995945 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:44.997605 (+ 1660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:44.998160 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:44.998167 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:44.998169 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:44.998276 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:44.998691 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:44.998695 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:44.998697 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:44.998941 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:44.999201 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:45.000548 (+ 1347us) server_negotiation.cc:299] Negotiation successful
0929 10:37:45.000665 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32,"spinlock_wait_cycles":6528}
I20260929 10:37:45.000844 5938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:44.987381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.124:44201 (local address 127.0.0.1:49624)
0929 10:37:44.987537 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:44.987551 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 10:37:44.987624 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:37:44.988374 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:37:44.988377 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:37:44.988542 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:37:44.988634 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:44.988644 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:44.989947 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:44.989951 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:44.990693 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:44.990704 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:44.991729 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:44.991734 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:44.991829 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:44.992244 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:37:44.992257 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:37:44.994441 (+ 2184us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:37:44.998397 (+ 3956us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:37:44.998400 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:37:44.998402 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:37:44.998711 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:37:44.999707 (+ 996us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:37:44.999710 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:37:44.999713 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:37:44.999811 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:37:45.000265 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:37:45.000268 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:37:45.000367 (+ 99us) client_negotiation.cc:769] Sending connection context
0929 10:37:45.000468 (+ 101us) client_negotiation.cc:240] Negotiation successful
0929 10:37:45.000601 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91}
I20260929 10:37:45.000864 6091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:44.987841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49624 (local address 127.15.82.124:44201)
0929 10:37:44.988143 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:44.988150 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:44.988162 (+ 12us) server_negotiation.cc:407] Connection header received
0929 10:37:44.988191 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:44.988193 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:44.988229 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:44.988744 (+ 515us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:44.988823 (+ 79us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:44.989833 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:44.990812 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:44.991588 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:44.992047 (+ 459us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:44.994600 (+ 2553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:44.994607 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:44.994611 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:37:44.994643 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:44.998303 (+ 3660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:44.999508 (+ 1205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:44.999511 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:44.999513 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:44.999598 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:44.999933 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:44.999938 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:44.999943 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:45.000144 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:45.000247 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:45.000554 (+ 307us) server_negotiation.cc:299] Negotiation successful
0929 10:37:45.000663 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":120,"threads_started":1}
I20260929 10:37:45.002504 6010 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a992d3ad0daa4078b50343e6463575a8: Bootstrap starting.
I20260929 10:37:45.003131 6016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:44.988303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57950 (local address 127.15.82.125:38327)
0929 10:37:44.988421 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:44.988426 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:44.988584 (+ 158us) server_negotiation.cc:407] Connection header received
0929 10:37:44.989031 (+ 447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:44.989035 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:44.989086 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:44.989142 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:44.990212 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:44.991128 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:44.993313 (+ 2185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:44.993774 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:44.993910 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:44.998684 (+ 4774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:44.998688 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:44.998690 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:37:44.998713 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:45.000410 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:45.001643 (+ 1233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:45.001647 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:45.001649 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:45.001730 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:45.002173 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:45.002176 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:45.002178 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:45.002441 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:45.002515 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:45.002874 (+ 359us) server_negotiation.cc:299] Negotiation successful
0929 10:37:45.002990 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":51,"spinlock_wait_cycles":1792}
I20260929 10:37:45.003544 6088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:44.988191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.125:38327 (local address 127.0.0.1:57950)
0929 10:37:44.988512 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:44.988528 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 10:37:44.988897 (+ 369us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:37:44.989312 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:37:44.989315 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:37:44.989903 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:37:44.990048 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:44.990058 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:44.991507 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:44.991510 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:44.993171 (+ 1661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:44.993181 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:44.993910 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:44.993914 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:44.994013 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:44.994747 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:37:44.994761 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:37:44.998522 (+ 3761us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:37:45.001130 (+ 2608us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:37:45.001134 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:37:45.001136 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:37:45.001526 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:37:45.001957 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:37:45.001962 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:37:45.001964 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:37:45.002056 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:37:45.002593 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:37:45.002596 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:37:45.002684 (+ 88us) client_negotiation.cc:769] Sending connection context
0929 10:37:45.003267 (+ 583us) client_negotiation.cc:240] Negotiation successful
0929 10:37:45.003400 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249}
I20260929 10:37:45.004117 5935 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70255adb66914018a8b1a4addb520a38: Bootstrap starting.
I20260929 10:37:45.004176 6086 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.82.124" port: 44201 } has no permanent_uuid. Determining permanent_uuid...
I20260929 10:37:45.006247 6010 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a992d3ad0daa4078b50343e6463575a8: Neither blocks nor log segments found. Creating new log.
I20260929 10:37:45.006734 5935 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 70255adb66914018a8b1a4addb520a38: Neither blocks nor log segments found. Creating new log.
I20260929 10:37:45.007182 6010 log.cc:824] T 00000000000000000000000000000000 P a992d3ad0daa4078b50343e6463575a8: Log is configured to *not* fsync() on all Append() calls
I20260929 10:37:45.007572 5935 log.cc:824] T 00000000000000000000000000000000 P 70255adb66914018a8b1a4addb520a38: Log is configured to *not* fsync() on all Append() calls
I20260929 10:37:45.009459 6010 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a992d3ad0daa4078b50343e6463575a8: No bootstrap required, opened a new log
I20260929 10:37:45.009648 5935 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70255adb66914018a8b1a4addb520a38: No bootstrap required, opened a new log
Sep 29 10:37:45 dist-test-slave-vg89 krb5kdc[5854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678264, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.82.124@KRBTEST.COM for kudu/127.15.82.124@KRBTEST.COM
Sep 29 10:37:45 dist-test-slave-vg89 krb5kdc[5854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678264, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.82.125@KRBTEST.COM
I20260929 10:37:45.013906 5935 raft_consensus.cc:348] T 00000000000000000000000000000000 P 70255adb66914018a8b1a4addb520a38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70255adb66914018a8b1a4addb520a38" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 45357 } } peers { permanent_uuid: "a992d3ad0daa4078b50343e6463575a8" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 38327 } } peers { permanent_uuid: "d4d7a827cb594bfead942e2d659eb116" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 44201 } }
I20260929 10:37:45.014150 5935 raft_consensus.cc:729] T 00000000000000000000000000000000 P 70255adb66914018a8b1a4addb520a38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70255adb66914018a8b1a4addb520a38, State: Initialized, Role: FOLLOWER
I20260929 10:37:45.014196 6010 raft_consensus.cc:348] T 00000000000000000000000000000000 P a992d3ad0daa4078b50343e6463575a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70255adb66914018a8b1a4addb520a38" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 45357 } } peers { permanent_uuid: "a992d3ad0daa4078b50343e6463575a8" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 38327 } } peers { permanent_uuid: "d4d7a827cb594bfead942e2d659eb116" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 44201 } }
I20260929 10:37:45.014540 6010 raft_consensus.cc:729] T 00000000000000000000000000000000 P a992d3ad0daa4078b50343e6463575a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a992d3ad0daa4078b50343e6463575a8, State: Initialized, Role: FOLLOWER
I20260929 10:37:45.014545 5935 consensus_queue.cc:260] T 00000000000000000000000000000000 P 70255adb66914018a8b1a4addb520a38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70255adb66914018a8b1a4addb520a38" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 45357 } } peers { permanent_uuid: "a992d3ad0daa4078b50343e6463575a8" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 38327 } } peers { permanent_uuid: "d4d7a827cb594bfead942e2d659eb116" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 44201 } }
I20260929 10:37:45.014881 6010 consensus_queue.cc:260] T 00000000000000000000000000000000 P a992d3ad0daa4078b50343e6463575a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70255adb66914018a8b1a4addb520a38" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 45357 } } peers { permanent_uuid: "a992d3ad0daa4078b50343e6463575a8" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 38327 } } peers { permanent_uuid: "d4d7a827cb594bfead942e2d659eb116" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 44201 } }
I20260929 10:37:45.015169 6016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:45.003754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57964 (local address 127.15.82.125:38327)
0929 10:37:45.003854 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:45.003858 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:45.004163 (+ 305us) server_negotiation.cc:407] Connection header received
0929 10:37:45.004426 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:45.004429 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:45.004472 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:45.004528 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:45.005401 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:45.006192 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:45.007498 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:45.007811 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:45.007890 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:45.011135 (+ 3245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:45.011140 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:45.011144 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:37:45.011171 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:45.012875 (+ 1704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:45.013355 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:45.013359 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:45.013360 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:45.013445 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:45.013741 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:45.013744 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:45.013746 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:45.013974 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:45.014147 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:45.014898 (+ 751us) server_negotiation.cc:299] Negotiation successful
0929 10:37:45.015022 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260929 10:37:45.015492 6088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:45.004623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.82.124:44201 (local address 127.0.0.1:49628)
0929 10:37:45.004946 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0929 10:37:45.004958 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:37:45.005027 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:37:45.005284 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:37:45.005286 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:37:45.005437 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:37:45.005514 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:45.005520 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:45.006463 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:45.006468 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:45.007239 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:37:45.007249 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:37:45.007919 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:37:45.007924 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:37:45.008003 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:45.008668 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:37:45.008687 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:37:45.011375 (+ 2688us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:37:45.013957 (+ 2582us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:37:45.013960 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:37:45.013962 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:37:45.014196 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:37:45.014490 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:37:45.014495 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:37:45.014498 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:37:45.014587 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:37:45.015011 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:37:45.015022 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:37:45.015105 (+ 83us) client_negotiation.cc:769] Sending connection context
0929 10:37:45.015210 (+ 105us) client_negotiation.cc:240] Negotiation successful
0929 10:37:45.015348 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284}
I20260929 10:37:45.015987 6091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:45.004804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49628 (local address 127.15.82.124:44201)
0929 10:37:45.004892 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:45.004896 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:45.005093 (+ 197us) server_negotiation.cc:407] Connection header received
0929 10:37:45.005116 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:45.005118 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:45.005149 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:45.005201 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:45.005623 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:45.006330 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:45.007369 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:45.007695 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:45.007807 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:45.011849 (+ 4042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:45.011853 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:45.011855 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:37:45.011879 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:45.013847 (+ 1968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:45.014300 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:45.014303 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:45.014304 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:45.014377 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:45.014685 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:45.014687 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:45.014689 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:45.014886 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:45.014977 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:45.015736 (+ 759us) server_negotiation.cc:299] Negotiation successful
0929 10:37:45.015857 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260929 10:37:45.016057 5935 sys_catalog.cc:564] T 00000000000000000000000000000000 P 70255adb66914018a8b1a4addb520a38 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:37:45.016950 6095 sys_catalog.cc:455] T 00000000000000000000000000000000 P a992d3ad0daa4078b50343e6463575a8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70255adb66914018a8b1a4addb520a38" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 45357 } } peers { permanent_uuid: "a992d3ad0daa4078b50343e6463575a8" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 38327 } } peers { permanent_uuid: "d4d7a827cb594bfead942e2d659eb116" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 44201 } } }
I20260929 10:37:45.017201 6095 sys_catalog.cc:458] T 00000000000000000000000000000000 P a992d3ad0daa4078b50343e6463575a8 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 10:37:45.017714 6010 sys_catalog.cc:564] T 00000000000000000000000000000000 P a992d3ad0daa4078b50343e6463575a8 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:37:45.018934 6094 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70255adb66914018a8b1a4addb520a38 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70255adb66914018a8b1a4addb520a38" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 45357 } } peers { permanent_uuid: "a992d3ad0daa4078b50343e6463575a8" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 38327 } } peers { permanent_uuid: "d4d7a827cb594bfead942e2d659eb116" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 44201 } } }
I20260929 10:37:45.019081 6094 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70255adb66914018a8b1a4addb520a38 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 10:37:45.019110 6086 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4d7a827cb594bfead942e2d659eb116: Bootstrap starting.
I20260929 10:37:45.020388 15689 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 10:37:45.021581 5909 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "70255adb66914018a8b1a4addb520a38"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41812
I20260929 10:37:45.021808 5909 raft_consensus.cc:482] T 00000000000000000000000000000000 P 70255adb66914018a8b1a4addb520a38 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
W20260929 10:37:45.021819 6106 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 70255adb66914018a8b1a4addb520a38: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260929 10:37:45.021884 6086 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d4d7a827cb594bfead942e2d659eb116: Neither blocks nor log segments found. Creating new log.
W20260929 10:37:45.021935 6106 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 10:37:45.021956 5909 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 70255adb66914018a8b1a4addb520a38 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:37:45.022811 6086 log.cc:824] T 00000000000000000000000000000000 P d4d7a827cb594bfead942e2d659eb116: Log is configured to *not* fsync() on all Append() calls
I20260929 10:37:45.023343 5909 raft_consensus.cc:504] T 00000000000000000000000000000000 P 70255adb66914018a8b1a4addb520a38 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70255adb66914018a8b1a4addb520a38" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 45357 } } peers { permanent_uuid: "a992d3ad0daa4078b50343e6463575a8" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 38327 } } peers { permanent_uuid: "d4d7a827cb594bfead942e2d659eb116" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 44201 } }
I20260929 10:37:45.024201 5909 leader_election.cc:288] T 00000000000000000000000000000000 P 70255adb66914018a8b1a4addb520a38 [CANDIDATE]: Term 1 election: Requested vote from peers a992d3ad0daa4078b50343e6463575a8 (127.15.82.125:38327), d4d7a827cb594bfead942e2d659eb116 (127.15.82.124:44201)
I20260929 10:37:45.025068 5984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "70255adb66914018a8b1a4addb520a38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "a992d3ad0daa4078b50343e6463575a8"
I20260929 10:37:45.025292 5984 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a992d3ad0daa4078b50343e6463575a8 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:37:45.026702 5984 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a992d3ad0daa4078b50343e6463575a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70255adb66914018a8b1a4addb520a38 in term 1.
I20260929 10:37:45.027108 5885 leader_election.cc:302] T 00000000000000000000000000000000 P 70255adb66914018a8b1a4addb520a38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 70255adb66914018a8b1a4addb520a38, a992d3ad0daa4078b50343e6463575a8; no voters:
I20260929 10:37:45.027374 6094 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 70255adb66914018a8b1a4addb520a38 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:37:45.027626 6094 raft_consensus.cc:686] T 00000000000000000000000000000000 P 70255adb66914018a8b1a4addb520a38 [term 1 LEADER]: Becoming Leader. State: Replica: 70255adb66914018a8b1a4addb520a38, State: Running, Role: LEADER
I20260929 10:37:45.027954 6094 consensus_queue.cc:237] T 00000000000000000000000000000000 P 70255adb66914018a8b1a4addb520a38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70255adb66914018a8b1a4addb520a38" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 45357 } } peers { permanent_uuid: "a992d3ad0daa4078b50343e6463575a8" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 38327 } } peers { permanent_uuid: "d4d7a827cb594bfead942e2d659eb116" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 44201 } }
I20260929 10:37:45.029714 6094 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70255adb66914018a8b1a4addb520a38 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 70255adb66914018a8b1a4addb520a38. Latest consensus state: current_term: 1 leader_uuid: "70255adb66914018a8b1a4addb520a38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70255adb66914018a8b1a4addb520a38" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 45357 } } peers { permanent_uuid: "a992d3ad0daa4078b50343e6463575a8" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 38327 } } peers { permanent_uuid: "d4d7a827cb594bfead942e2d659eb116" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 44201 } } }
I20260929 10:37:45.029820 6094 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70255adb66914018a8b1a4addb520a38 [sys.catalog]: This master's current role is: LEADER
I20260929 10:37:45.029014 5867 master_runner.cc:186] Error getting master registration for 127.15.82.124:44201: OK, instance_id { permanent_uuid: "d4d7a827cb594bfead942e2d659eb116" instance_seqno: 1790678264957241 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 10:37:45.032774 5867 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 10:37:45.035504 6129 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:37:45.036517 6129 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:37:45.036995 5942 master_runner.cc:186] Error getting master registration for 127.15.82.124:44201: OK, instance_id { permanent_uuid: "d4d7a827cb594bfead942e2d659eb116" instance_seqno: 1790678264957241 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 10:37:45.040957 5942 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260929 10:37:45.041692 6119 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a992d3ad0daa4078b50343e6463575a8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 10:37:45.041774 6119 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 10:37:45.043538 6091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:45.028410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49644 (local address 127.15.82.124:44201)
0929 10:37:45.028513 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:45.028518 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:45.030638 (+ 2120us) server_negotiation.cc:407] Connection header received
0929 10:37:45.030799 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:45.030802 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:45.030845 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:45.030894 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:45.031819 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:45.032580 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:45.033811 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:45.034173 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:45.034279 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:45.036237 (+ 1958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:45.036241 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:45.036244 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:37:45.036270 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:45.037994 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:45.041922 (+ 3928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:45.041926 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:45.041928 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:45.042025 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:45.042369 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:45.042375 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:45.042378 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:45.042671 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:45.042807 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:45.043194 (+ 387us) server_negotiation.cc:299] Negotiation successful
0929 10:37:45.043350 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260929 10:37:45.043867 5983 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a992d3ad0daa4078b50343e6463575a8 [term 1 FOLLOWER]: Refusing update from remote peer 70255adb66914018a8b1a4addb520a38: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:37:45.044597 6094 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 70255adb66914018a8b1a4addb520a38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a992d3ad0daa4078b50343e6463575a8" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 38327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:37:45.046218 6016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:45.027953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57972 (local address 127.15.82.125:38327)
0929 10:37:45.028068 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:45.028072 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:45.029664 (+ 1592us) server_negotiation.cc:407] Connection header received
0929 10:37:45.029846 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:45.029851 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:45.029900 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:45.029950 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:45.034482 (+ 4532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:45.035248 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:45.037152 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:45.037508 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:45.037581 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:45.042197 (+ 4616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:45.042203 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:45.042205 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:37:45.042230 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:45.043955 (+ 1725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:45.044725 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:45.044730 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:45.044731 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:45.044834 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:45.045252 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:45.045256 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:45.045258 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:45.045482 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:45.045603 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:45.045956 (+ 353us) server_negotiation.cc:299] Negotiation successful
0929 10:37:45.046079 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260929 10:37:45.046818 6086 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4d7a827cb594bfead942e2d659eb116: No bootstrap required, opened a new log
I20260929 10:37:45.049579 6095 sys_catalog.cc:455] T 00000000000000000000000000000000 P a992d3ad0daa4078b50343e6463575a8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 70255adb66914018a8b1a4addb520a38. Latest consensus state: current_term: 1 leader_uuid: "70255adb66914018a8b1a4addb520a38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70255adb66914018a8b1a4addb520a38" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 45357 } } peers { permanent_uuid: "a992d3ad0daa4078b50343e6463575a8" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 38327 } } peers { permanent_uuid: "d4d7a827cb594bfead942e2d659eb116" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 44201 } } }
I20260929 10:37:45.049695 6095 sys_catalog.cc:458] T 00000000000000000000000000000000 P a992d3ad0daa4078b50343e6463575a8 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 10:37:45.051728 6086 raft_consensus.cc:348] T 00000000000000000000000000000000 P d4d7a827cb594bfead942e2d659eb116 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70255adb66914018a8b1a4addb520a38" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 45357 } } peers { permanent_uuid: "a992d3ad0daa4078b50343e6463575a8" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 38327 } } peers { permanent_uuid: "d4d7a827cb594bfead942e2d659eb116" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 44201 } }
I20260929 10:37:45.051980 6086 raft_consensus.cc:729] T 00000000000000000000000000000000 P d4d7a827cb594bfead942e2d659eb116 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4d7a827cb594bfead942e2d659eb116, State: Initialized, Role: FOLLOWER
I20260929 10:37:45.052338 6086 consensus_queue.cc:260] T 00000000000000000000000000000000 P d4d7a827cb594bfead942e2d659eb116 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70255adb66914018a8b1a4addb520a38" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 45357 } } peers { permanent_uuid: "a992d3ad0daa4078b50343e6463575a8" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 38327 } } peers { permanent_uuid: "d4d7a827cb594bfead942e2d659eb116" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 44201 } }
I20260929 10:37:45.052884 6134 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4d7a827cb594bfead942e2d659eb116 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70255adb66914018a8b1a4addb520a38" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 45357 } } peers { permanent_uuid: "a992d3ad0daa4078b50343e6463575a8" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 38327 } } peers { permanent_uuid: "d4d7a827cb594bfead942e2d659eb116" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 44201 } } }
I20260929 10:37:45.053153 6134 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4d7a827cb594bfead942e2d659eb116 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 10:37:45.053766 6086 sys_catalog.cc:564] T 00000000000000000000000000000000 P d4d7a827cb594bfead942e2d659eb116 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:37:45.053866 6060 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "70255adb66914018a8b1a4addb520a38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "d4d7a827cb594bfead942e2d659eb116"
I20260929 10:37:45.054008 6059 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d4d7a827cb594bfead942e2d659eb116 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:37:45.055676 6059 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d4d7a827cb594bfead942e2d659eb116 [term 1 FOLLOWER]: Refusing update from remote peer 70255adb66914018a8b1a4addb520a38: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:37:45.056432 6094 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 70255adb66914018a8b1a4addb520a38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4d7a827cb594bfead942e2d659eb116" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 44201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:37:45.059304 5939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:45.029335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41818 (local address 127.15.82.126:45357)
0929 10:37:45.030035 (+ 700us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:45.030040 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:45.034988 (+ 4948us) server_negotiation.cc:407] Connection header received
0929 10:37:45.035119 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:45.035121 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:45.035166 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:45.035240 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:45.039463 (+ 4223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:45.040173 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:45.043006 (+ 2833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:45.043324 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:45.043405 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:45.054640 (+ 11235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:37:45.054644 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:37:45.054647 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:37:45.054687 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:37:45.056629 (+ 1942us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:45.057244 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:45.057248 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:45.057257 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:45.057357 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:37:45.057709 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:37:45.057716 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:37:45.057720 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:37:45.057992 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:37:45.058125 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:45.059004 (+ 879us) server_negotiation.cc:299] Negotiation successful
0929 10:37:45.059138 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":633}
I20260929 10:37:45.057875 6060 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d4d7a827cb594bfead942e2d659eb116 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70255adb66914018a8b1a4addb520a38 in term 1.
I20260929 10:37:45.064713 6124 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70255adb66914018a8b1a4addb520a38 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "70255adb66914018a8b1a4addb520a38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70255adb66914018a8b1a4addb520a38" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 45357 } } peers { permanent_uuid: "a992d3ad0daa4078b50343e6463575a8" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 38327 } } peers { permanent_uuid: "d4d7a827cb594bfead942e2d659eb116" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 44201 } } }
I20260929 10:37:45.064855 6124 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70255adb66914018a8b1a4addb520a38 [sys.catalog]: This master's current role is: LEADER
I20260929 10:37:45.065192 6132 mvcc.cc:205] Tried to move back new op lower bound from 7334618173609345024 to 7334618173557936128. Current Snapshot: MvccSnapshot[applied={T|T < 7334618173609345024}]
I20260929 10:37:45.065336 6095 sys_catalog.cc:455] T 00000000000000000000000000000000 P a992d3ad0daa4078b50343e6463575a8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "70255adb66914018a8b1a4addb520a38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70255adb66914018a8b1a4addb520a38" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 45357 } } peers { permanent_uuid: "a992d3ad0daa4078b50343e6463575a8" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 38327 } } peers { permanent_uuid: "d4d7a827cb594bfead942e2d659eb116" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 44201 } } }
I20260929 10:37:45.065438 6095 sys_catalog.cc:458] T 00000000000000000000000000000000 P a992d3ad0daa4078b50343e6463575a8 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 10:37:45.066646 6134 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4d7a827cb594bfead942e2d659eb116 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 70255adb66914018a8b1a4addb520a38. Latest consensus state: current_term: 1 leader_uuid: "70255adb66914018a8b1a4addb520a38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70255adb66914018a8b1a4addb520a38" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 45357 } } peers { permanent_uuid: "a992d3ad0daa4078b50343e6463575a8" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 38327 } } peers { permanent_uuid: "d4d7a827cb594bfead942e2d659eb116" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 44201 } } }
I20260929 10:37:45.066789 6134 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4d7a827cb594bfead942e2d659eb116 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 10:37:45.066955 6129 catalog_manager.cc:1347] Generated new cluster ID: 118dbc0a93a140f9979fc121de24fbf2
I20260929 10:37:45.067018 6129 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
W20260929 10:37:45.069638 6145 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d4d7a827cb594bfead942e2d659eb116: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 10:37:45.069707 6145 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 10:37:45.087045 6129 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:37:45.088315 6129 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:37:45.090152 6124 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70255adb66914018a8b1a4addb520a38 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "70255adb66914018a8b1a4addb520a38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70255adb66914018a8b1a4addb520a38" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 45357 } } peers { permanent_uuid: "a992d3ad0daa4078b50343e6463575a8" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 38327 } } peers { permanent_uuid: "d4d7a827cb594bfead942e2d659eb116" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 44201 } } }
I20260929 10:37:45.090293 6124 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70255adb66914018a8b1a4addb520a38 [sys.catalog]: This master's current role is: LEADER
I20260929 10:37:45.091740 6134 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4d7a827cb594bfead942e2d659eb116 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "70255adb66914018a8b1a4addb520a38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70255adb66914018a8b1a4addb520a38" member_type: VOTER last_known_addr { host: "127.15.82.126" port: 45357 } } peers { permanent_uuid: "a992d3ad0daa4078b50343e6463575a8" member_type: VOTER last_known_addr { host: "127.15.82.125" port: 38327 } } peers { permanent_uuid: "d4d7a827cb594bfead942e2d659eb116" member_type: VOTER last_known_addr { host: "127.15.82.124" port: 44201 } } }
I20260929 10:37:45.091863 6134 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4d7a827cb594bfead942e2d659eb116 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 10:37:45.091967 6147 mvcc.cc:205] Tried to move back new op lower bound from 7334618173609345024 to 7334618173557936128. Current Snapshot: MvccSnapshot[applied={T|T < 7334618173609345024}]
I20260929 10:37:45.111306 6129 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 70255adb66914018a8b1a4addb520a38: Generated new TSK 0
I20260929 10:37:45.112237 6129 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260929 10:37:45.146260 6016 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:37:45.142453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57988 (local address 127.15.82.125:38327)
0929 10:37:45.142592 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:45.142597 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:45.144510 (+ 1913us) server_negotiation.cc:407] Connection header received
0929 10:37:45.145041 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:45.145046 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:45.145115 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:45.145196 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:45.146106 (+ 910us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57988: BlockingRecv error: recv got EOF from 127.0.0.1:57988 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260929 10:37:45.147773 6091 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:37:45.143353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49648 (local address 127.15.82.124:44201)
0929 10:37:45.143475 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:45.143479 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:45.146752 (+ 3273us) server_negotiation.cc:407] Connection header received
0929 10:37:45.146945 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:45.146949 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:45.147004 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:45.147055 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:45.147646 (+ 591us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49648: BlockingRecv error: recv got EOF from 127.0.0.1:49648 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260929 10:37:45.149447 5939 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:37:45.144125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41834 (local address 127.15.82.126:45357)
0929 10:37:45.144243 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:45.144248 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:45.148332 (+ 4084us) server_negotiation.cc:407] Connection header received
0929 10:37:45.148660 (+ 328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:45.148663 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:45.148720 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:45.148789 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:45.149318 (+ 529us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41834: BlockingRecv error: recv got EOF from 127.0.0.1:41834 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260929 10:37:45.154095 6016 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:37:45.152493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57992 (local address 127.15.82.125:38327)
0929 10:37:45.152601 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:45.152605 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:45.153055 (+ 450us) server_negotiation.cc:407] Connection header received
0929 10:37:45.153279 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:45.153282 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:45.153380 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:45.153440 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:45.153972 (+ 532us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57992: BlockingRecv error: recv got EOF from 127.0.0.1:57992 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260929 10:37:45.159477 6016 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:37:45.157850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57996 (local address 127.15.82.125:38327)
0929 10:37:45.157955 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:45.157959 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:45.158595 (+ 636us) server_negotiation.cc:407] Connection header received
0929 10:37:45.158778 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:45.158781 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:45.158833 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:45.158885 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:45.159374 (+ 489us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57996: BlockingRecv error: recv got EOF from 127.0.0.1:57996 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260929 10:37:45.165117 6016 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:37:45.163552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58004 (local address 127.15.82.125:38327)
0929 10:37:45.163656 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:45.163660 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:45.164176 (+ 516us) server_negotiation.cc:407] Connection header received
0929 10:37:45.164378 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:45.164382 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:45.164438 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:45.164490 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:45.165006 (+ 516us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58004: BlockingRecv error: recv got EOF from 127.0.0.1:58004 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260929 10:37:45.172830 6016 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:37:45.171210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58018 (local address 127.15.82.125:38327)
0929 10:37:45.171370 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:45.171374 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:45.171902 (+ 528us) server_negotiation.cc:407] Connection header received
0929 10:37:45.172089 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:45.172093 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:45.172145 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:45.172196 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:45.172716 (+ 520us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58018: BlockingRecv error: recv got EOF from 127.0.0.1:58018 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
W20260929 10:37:45.185473 6016 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:37:45.183522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58030 (local address 127.15.82.125:38327)
0929 10:37:45.183676 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:45.183682 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:45.184357 (+ 675us) server_negotiation.cc:407] Connection header received
0929 10:37:45.184588 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:45.184592 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:45.184652 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:45.184710 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:45.185334 (+ 624us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58030: BlockingRecv error: recv got EOF from 127.0.0.1:58030 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260929 10:37:45.207329 6016 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:37:45.205224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58042 (local address 127.15.82.125:38327)
0929 10:37:45.205362 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:45.205367 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:45.206146 (+ 779us) server_negotiation.cc:407] Connection header received
0929 10:37:45.206369 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:45.206373 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:45.206456 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:45.206514 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:45.207202 (+ 688us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58042: BlockingRecv error: recv got EOF from 127.0.0.1:58042 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260929 10:37:45.243690 6016 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:37:45.241897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58044 (local address 127.15.82.125:38327)
0929 10:37:45.242021 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:45.242026 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:45.242665 (+ 639us) server_negotiation.cc:407] Connection header received
0929 10:37:45.242886 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:45.242889 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:45.242947 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:45.243001 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:45.243563 (+ 562us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58044: BlockingRecv error: recv got EOF from 127.0.0.1:58044 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260929 10:37:45.311710 6016 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:37:45.310322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58058 (local address 127.15.82.125:38327)
0929 10:37:45.310509 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:45.310514 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:45.310713 (+ 199us) server_negotiation.cc:407] Connection header received
0929 10:37:45.310881 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:45.310884 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:45.310937 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:45.311014 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:45.311584 (+ 570us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58058: BlockingRecv error: recv got EOF from 127.0.0.1:58058 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260929 10:37:45.443444 6016 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:37:45.442094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58072 (local address 127.15.82.125:38327)
0929 10:37:45.442260 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:45.442265 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:45.442580 (+ 315us) server_negotiation.cc:407] Connection header received
0929 10:37:45.442695 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:45.442697 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:45.442747 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:45.442839 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:45.443300 (+ 461us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58072: BlockingRecv error: recv got EOF from 127.0.0.1:58072 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260929 10:37:45.702980 6016 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:37:45.701790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58076 (local address 127.15.82.125:38327)
0929 10:37:45.701939 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:45.701946 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:45.702004 (+ 58us) server_negotiation.cc:407] Connection header received
0929 10:37:45.702199 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:45.702202 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:45.702255 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:45.702352 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:37:45.702848 (+ 496us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58076: BlockingRecv error: recv got EOF from 127.0.0.1:58076 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
I20260929 10:37:46.043114 6119 catalog_manager.cc:1259] Loaded cluster ID: 118dbc0a93a140f9979fc121de24fbf2
I20260929 10:37:46.043201 6119 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a992d3ad0daa4078b50343e6463575a8: loading cluster ID for follower catalog manager: success
I20260929 10:37:46.044833 6119 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a992d3ad0daa4078b50343e6463575a8: acquiring CA information for follower catalog manager: success
I20260929 10:37:46.045941 6119 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a992d3ad0daa4078b50343e6463575a8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 10:37:46.071066 6145 catalog_manager.cc:1259] Loaded cluster ID: 118dbc0a93a140f9979fc121de24fbf2
I20260929 10:37:46.071153 6145 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d4d7a827cb594bfead942e2d659eb116: loading cluster ID for follower catalog manager: success
I20260929 10:37:46.072870 6145 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d4d7a827cb594bfead942e2d659eb116: acquiring CA information for follower catalog manager: success
I20260929 10:37:46.074060 6145 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d4d7a827cb594bfead942e2d659eb116: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 10:37:46.221799 6218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:46.217398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58086 (local address 127.15.82.125:38327)
0929 10:37:46.217671 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:46.217678 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:46.217782 (+ 104us) server_negotiation.cc:407] Connection header received
0929 10:37:46.217967 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:46.217975 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:46.218173 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:46.218332 (+ 159us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:37:46.218668 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:46.219260 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:46.220183 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:46.220442 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:46.220563 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:46.220922 (+ 359us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:37:46.221361 (+ 439us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:37:46.221506 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:46.221554 (+ 48us) server_negotiation.cc:299] Negotiation successful
0929 10:37:46.221615 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":77,"threads_started":1}
I20260929 10:37:46.228821 6224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:46.224908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49650 (local address 127.15.82.124:44201)
0929 10:37:46.225135 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:46.225138 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:46.225150 (+ 12us) server_negotiation.cc:407] Connection header received
0929 10:37:46.225225 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:46.225227 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:46.225342 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:46.225438 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:37:46.225654 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:46.226141 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:46.227130 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:46.227374 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:46.227467 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:46.227785 (+ 318us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:37:46.228221 (+ 436us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:37:46.228351 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:46.228648 (+ 297us) server_negotiation.cc:299] Negotiation successful
0929 10:37:46.228701 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":93,"threads_started":1}
I20260929 10:37:46.236740 6218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:46.232201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58090 (local address 127.15.82.125:38327)
0929 10:37:46.232356 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:46.232362 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:46.232906 (+ 544us) server_negotiation.cc:407] Connection header received
0929 10:37:46.233023 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:46.233026 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:46.233078 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:46.233143 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:37:46.233520 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:46.234088 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:46.235446 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:46.235785 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:46.235873 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:46.236239 (+ 366us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:37:46.236327 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:37:46.236396 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:46.236517 (+ 121us) server_negotiation.cc:299] Negotiation successful
0929 10:37:46.236571 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260929 10:37:46.236758 6224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:46.232739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49656 (local address 127.15.82.124:44201)
0929 10:37:46.232896 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:46.232902 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:46.232918 (+ 16us) server_negotiation.cc:407] Connection header received
0929 10:37:46.233084 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:46.233087 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:46.233146 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:46.233216 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:37:46.233466 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:46.234220 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:46.235412 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:46.235668 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:46.235782 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:46.236059 (+ 277us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:37:46.236138 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:37:46.236210 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:46.236556 (+ 346us) server_negotiation.cc:299] Negotiation successful
0929 10:37:46.236610 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260929 10:37:46.237162 6230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:37:46.232138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41838 (local address 127.15.82.126:45357)
0929 10:37:46.232424 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0929 10:37:46.232429 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:37:46.232536 (+ 107us) server_negotiation.cc:407] Connection header received
0929 10:37:46.232721 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:37:46.232724 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:37:46.233107 (+ 383us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:37:46.233256 (+ 149us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:37:46.233686 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:46.234505 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:46.235456 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:37:46.235807 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:37:46.235899 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:37:46.236221 (+ 322us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:37:46.236594 (+ 373us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:37:46.236670 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0929 10:37:46.236956 (+ 286us) server_negotiation.cc:299] Negotiation successful
0929 10:37:46.237017 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":118,"threads_started":1}
I20260929 10:37:46.240705 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 5867
I20260929 10:37:46.247522 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 5942
I20260929 10:37:46.253301 15689 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiKMUyb/build/debug/bin/kudu with pid 6017
2026-09-29T10:37:46Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3125 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6387 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (284649 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST